UPDATE - WebDNA - UNIX Shell program request

This WebDNA talk-list message is from

2004


It keeps the original formatting.
numero = 59335
interpreted = N
texte = What I discovered after I mailed the request was that I could directly import a fixed length file after all; the SQL manual online alludes to it, but the book I am using to help decipher SQL is in error on the matter. A user pointed me in the right direction, and after some experimentation, I got it to work as promised. Using the LOAD DATA LOCAL INTO FILE command with appropriate parameters resulted in a file parse; time less than a second. If anyone wants this code, let me know. As usual, I get to blame the documentation for my own shortcomings! Thanks to all who replied. Larry On Sep 9, 2004, at 9:11 AM, Larry Hewitt wrote: > I am looking for some UNIX experience. > > BUSINESS CASE: > > We run VoteWashington.Org, Washington State's county election > management service. OSX, WebCat 6, MySQL > > We receive an ascii file in fixed length format from a major county > directly out of their ballot counting equipment with precinct level > vote data; 28000 lines of data that needs to be parsed into a SQL > table. The client cannot reprogram the file output to include any > delimiters - limitations of the software in the elections equipment. > It s not acceptable to ask the client to intervene in the process to > do this manually on election night. > > They upload the file, and begin a process whereby the file is parsed > into usable form and read into tables. It is lengthy - according to > what I understand (a relative beginner with SQL), SQL cannot > automatically parse in a LOAD INTO FILE command a fixed-length file. > > I have a routine that does the parsing, but am looking for a more > efficient manner of processing the file. > > OPPORTUNITY: > > If anyone out there has experience with UNIX and OSX, I am wondering > if a shell script can be written and run from the WedDNA SHELL context > to parse the file at the OS level, inserting delimeters. The LOAD > INTO FILE could then be used without the following parsing routine. > > I am willing to pay for a successful implementation of this, or > something like it that resolves the issue of the long parsing routine. > > If anybody is interested in discussing this offline, please email me > directly. I can share the sample file with you, and give you some > more particulars about this issue. > > TIA > > LArry > > Larry Hewitt360.807.1100 OFFICE > HCO, LLC360.807.1103 FAX > 109 No. Tower360.880.4855 CELL > P.O. Box 1017larryh@hewittco.com > Centralia, WA 98531www.votewashington.org > www.hewittco.com > > > ------------------------------------------------------------- > This message is sent to you because you are subscribed to > the mailing list . > To unsubscribe, E-mail to: > To switch to the DIGEST mode, E-mail to > > Web Archive of this list is at: http://webdna.smithmicro.com/ > > Larry Hewitt360.807.1100 OFFICE HCO, LLC360.807.1103 FAX 109 No. Tower360.880.4855 CELL P.O. Box 1017larryh@hewittco.com Centralia, WA 98531www.votewashington.org www.hewittco.com ------------------------------------------------------------- This message is sent to you because you are subscribed to the mailing list . To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to Web Archive of this list is at: http://webdna.smithmicro.com/ Associated Messages, from the most recent to the oldest:

    
  1. UPDATE - WebDNA - UNIX Shell program request ( Larry Hewitt 2004)
What I discovered after I mailed the request was that I could directly import a fixed length file after all; the SQL manual online alludes to it, but the book I am using to help decipher SQL is in error on the matter. A user pointed me in the right direction, and after some experimentation, I got it to work as promised. Using the LOAD DATA LOCAL INTO FILE command with appropriate parameters resulted in a file parse; time less than a second. If anyone wants this code, let me know. As usual, I get to blame the documentation for my own shortcomings! Thanks to all who replied. Larry On Sep 9, 2004, at 9:11 AM, Larry Hewitt wrote: > I am looking for some UNIX experience. > > BUSINESS CASE: > > We run VoteWashington.Org, Washington State's county election > management service. OSX, WebCat 6, MySQL > > We receive an ascii file in fixed length format from a major county > directly out of their ballot counting equipment with precinct level > vote data; 28000 lines of data that needs to be parsed into a SQL > table. The client cannot reprogram the file output to include any > delimiters - limitations of the software in the elections equipment. > It s not acceptable to ask the client to intervene in the process to > do this manually on election night. > > They upload the file, and begin a process whereby the file is parsed > into usable form and read into tables. It is lengthy - according to > what I understand (a relative beginner with SQL), SQL cannot > automatically parse in a LOAD INTO FILE command a fixed-length file. > > I have a routine that does the parsing, but am looking for a more > efficient manner of processing the file. > > OPPORTUNITY: > > If anyone out there has experience with UNIX and OSX, I am wondering > if a shell script can be written and run from the WedDNA SHELL context > to parse the file at the OS level, inserting delimeters. The LOAD > INTO FILE could then be used without the following parsing routine. > > I am willing to pay for a successful implementation of this, or > something like it that resolves the issue of the long parsing routine. > > If anybody is interested in discussing this offline, please email me > directly. I can share the sample file with you, and give you some > more particulars about this issue. > > TIA > > LArry > > Larry Hewitt360.807.1100 OFFICE > HCO, LLC360.807.1103 FAX > 109 No. Tower360.880.4855 CELL > P.O. Box 1017larryh@hewittco.com > Centralia, WA 98531www.votewashington.org > www.hewittco.com > > > ------------------------------------------------------------- > This message is sent to you because you are subscribed to > the mailing list . > To unsubscribe, E-mail to: > To switch to the DIGEST mode, E-mail to > > Web Archive of this list is at: http://webdna.smithmicro.com/ > > Larry Hewitt360.807.1100 OFFICE HCO, LLC360.807.1103 FAX 109 No. Tower360.880.4855 CELL P.O. Box 1017larryh@hewittco.com Centralia, WA 98531www.votewashington.org www.hewittco.com ------------------------------------------------------------- This message is sent to you because you are subscribed to the mailing list . To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to Web Archive of this list is at: http://webdna.smithmicro.com/ Larry Hewitt

DOWNLOAD WEBDNA NOW!

Top Articles:

Talk List

The WebDNA community talk-list is the best place to get some help: several hundred extremely proficient programmers with an excellent knowledge of WebDNA and an excellent spirit will deliver all the tips and tricks you can imagine...

Related Readings:

Re Running Locally (2001) Quitting WebMerchant ? (1997) PCS Emailer's role ? (1997) target=_blank and form variables (1997) Show if time tags (1997) More Applescript (1997) Using [Include] Context (1999) fresh eyes needed. Append won't work. (2000) form crasehes server (1997) System 8.5.1 question (1999) Nested searches (1998) Another question (1997) [WebDNA] Hosting Needs for WebDNA (2014) Ongoing group search problems ... (1997) Projects & Contractors (1997) A few questions... (1999) Separate SSL Server (1997) Can you identify the [body]? (1999) RE: Signal Raised (1997) WebCat2b13MacPlugin - [math][date][/math] problem (1997)