Re: Slow Code, how to speed up

This WebDNA talk-list message is from

2002


It keeps the original formatting.
numero = 41119
interpreted = N
texte = >Here's code that takes 1600 ticks to process on my server. How can I speed it up? It is opening a 700k file and then processing it, and the file is getting larger at all times, since it is a list of referrering web sites. > >[!] Search Temp Database [/!] >[search db=referer.txt&neREFERRERdatarq=findall&REFERRERsumm=t][founditems] > >[!] Append to Temp 2 Database [/!] >[append db=templog2.db]referrer=[referrer]&count=[search >db=referer.txt&eqREFERRERdatarq=[url][REFERRER][/url]&deDTsort=1&DTtype=date >&max=1][numfound]&dt=[founditems][dt][/founditems][/search][/append] >[/founditems][/search]That inner [append] is taking more time because it is opening and closing the database file once per time through your outer [founditems] loop, and all that file access is killing the speed.Instead, open templog2.db just once outside the loop, then write raw data to the end of templog2.db using [appendfile], and then flush templog2.db to force it to be read back in as a database. Here's pseudo-code for it. You have to be *very* careful about your linebreaks here, because you don't want extra carriage returns in the middle of your templog.db file to screw up the db format:close templog2.db [text]tab=%09[/text] [appendfile file=templog2.db][search db=referer.txt&blah][founditems][referrer][tab][your count stuff][tab][your dt stuff] [/founditems][/appendfile]The idea here is that with [appendfile] you are just opening the file once, and doing all the writing to it inside your loop. -- ______________________________________________________________________________ Grant Hulbert | Internet Software Inventor | http://www.hulbertfamily.com/------------------------------------------------------------- This message is sent to you because you are subscribed to the mailing list . To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to Web Archive of this list is at: http://search.smithmicro.com/ Associated Messages, from the most recent to the oldest:

    
  1. Re: Slow Code, how to speed up (Charles Kline 2002)
  2. Re: Slow Code, how to speed up (Jesse Williams-Proudman 2002)
  3. Re: Slow Code, how to speed up (Grant Hulbert 2002)
  4. Re: Slow Code, how to speed up (Charles Kline 2002)
  5. Re: Slow Code, how to speed up (Charles Kline 2002)
  6. Re: Slow Code, how to speed up (Grant Hulbert 2002)
  7. Slow Code, how to speed up (Jeff Logan 2002)
>Here's code that takes 1600 ticks to process on my server. How can I speed it up? It is opening a 700k file and then processing it, and the file is getting larger at all times, since it is a list of referrering web sites. > >[!] Search Temp Database [/!] >[search db=referer.txt&neREFERRERdatarq=findall&REFERRERsumm=t][founditems] > >[!] Append to Temp 2 Database [/!] >[append db=templog2.db]referrer=[referrer]&count=[search >db=referer.txt&eqREFERRERdatarq=[url][referrer][/url]&deDTsort=1&DTtype=date >&max=1][numfound]&dt=[founditems][dt][/founditems][/search][/append] >[/founditems][/search]That inner [append] is taking more time because it is opening and closing the database file once per time through your outer [founditems] loop, and all that file access is killing the speed.Instead, open templog2.db just once outside the loop, then write raw data to the end of templog2.db using [appendfile], and then flush templog2.db to force it to be read back in as a database. Here's pseudo-code for it. You have to be *very* careful about your linebreaks here, because you don't want extra carriage returns in the middle of your templog.db file to screw up the db format:close templog2.db [text]tab=%09[/text] [appendfile file=templog2.db][search db=referer.txt&blah][founditems][referrer][tab][your count stuff][tab][your dt stuff] [/founditems][/appendfile]The idea here is that with [appendfile] you are just opening the file once, and doing all the writing to it inside your loop. -- ______________________________________________________________________________ Grant Hulbert | Internet Software Inventor | http://www.hulbertfamily.com/------------------------------------------------------------- This message is sent to you because you are subscribed to the mailing list . To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to Web Archive of this list is at: http://search.smithmicro.com/ Grant Hulbert

DOWNLOAD WEBDNA NOW!

Top Articles:

Talk List

The WebDNA community talk-list is the best place to get some help: several hundred extremely proficient programmers with an excellent knowledge of WebDNA and an excellent spirit will deliver all the tips and tricks you can imagine...

Related Readings:

NT considerations (1997) [WebDNA] Page names changing (2010) Mac OS and 3.0.5b12 bug report (1999) Getting Rid of Multiple Returns (2001) Emailer setup (1997) Search results templates (1996) Am I going senile? (Price recalc based on quantity) (1997) For those of you not on the WebCatalog Beta... (1997) [OT] DOD again (2003) breaking words (1997) Custom formulas.db (1998) absolute paths for databases? (1997) quantity minimum problem (1997) Form Tempelate (2000) Secure Server (1997) WCS Newbie question (1997) Your Pay (2004) [writefile] (1997) More questions about serial number dishing (1997) Frames and WebCat (1997)