Re: Slow Code, how to speed up
This WebDNA talk-list message is from 2002
It keeps the original formatting.
numero = 41119
interpreted = N
texte = >Here's code that takes 1600 ticks to process on my server. How can I speed it up? It is opening a 700k file and then processing it, and the file is getting larger at all times, since it is a list of referrering web sites. > >[!] Search Temp Database [/!] >[search db=referer.txt&neREFERRERdatarq=findall&REFERRERsumm=t][founditems] > >[!] Append to Temp 2 Database [/!] >[append db=templog2.db]referrer=[referrer]&count=[search >db=referer.txt&eqREFERRERdatarq=[url][REFERRER][/url]&deDTsort=1&DTtype=date >&max=1][numfound]&dt=[founditems][dt][/founditems][/search][/append] >[/founditems][/search]That inner [append] is taking more time because it is opening and closing the database file once per time through your outer [founditems] loop, and all that file access is killing the speed.Instead, open templog2.db just once outside the loop, then write raw data to the end of templog2.db using [appendfile], and then flush templog2.db to force it to be read back in as a database. Here's pseudo-code for it. You have to be *very* careful about your linebreaks here, because you don't want extra carriage returns in the middle of your templog.db file to screw up the db format:close templog2.db [text]tab=%09[/text] [appendfile file=templog2.db][search db=referer.txt&blah][founditems][referrer][tab][your count stuff][tab][your dt stuff] [/founditems][/appendfile]The idea here is that with [appendfile] you are just opening the file once, and doing all the writing to it inside your loop. -- ______________________________________________________________________________ Grant Hulbert | Internet Software Inventor | http://www.hulbertfamily.com/------------------------------------------------------------- This message is sent to you because you are subscribed to the mailing list
. To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to Web Archive of this list is at: http://search.smithmicro.com/
Associated Messages, from the most recent to the oldest:
>Here's code that takes 1600 ticks to process on my server. How can I speed it up? It is opening a 700k file and then processing it, and the file is getting larger at all times, since it is a list of referrering web sites. > >[!] Search Temp Database [/!] >[search db=referer.txt&neREFERRERdatarq=findall&REFERRERsumm=t][founditems] > >[!] Append to Temp 2 Database [/!] >[append db=templog2.db]referrer=[referrer]&count=[search >db=referer.txt&eqREFERRERdatarq=[url][referrer][/url]&deDTsort=1&DTtype=date >&max=1][numfound]&dt=[founditems][dt][/founditems][/search][/append] >[/founditems][/search]That inner [append] is taking more time because it is opening and closing the database file once per time through your outer [founditems] loop, and all that file access is killing the speed.Instead, open templog2.db just once outside the loop, then write raw data to the end of templog2.db using [appendfile], and then flush templog2.db to force it to be read back in as a database. Here's pseudo-code for it. You have to be *very* careful about your linebreaks here, because you don't want extra carriage returns in the middle of your templog.db file to screw up the db format:close templog2.db [text]tab=%09[/text] [appendfile file=templog2.db][search db=referer.txt&blah][founditems][referrer][tab][your count stuff][tab][your dt stuff] [/founditems][/appendfile]The idea here is that with [appendfile] you are just opening the file once, and doing all the writing to it inside your loop. -- ______________________________________________________________________________ Grant Hulbert | Internet Software Inventor | http://www.hulbertfamily.com/------------------------------------------------------------- This message is sent to you because you are subscribed to the mailing list . To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to Web Archive of this list is at: http://search.smithmicro.com/
Grant Hulbert
DOWNLOAD WEBDNA NOW!
Top Articles:
Talk List
The WebDNA community talk-list is the best place to get some help: several hundred extremely proficient programmers with an excellent knowledge of WebDNA and an excellent spirit will deliver all the tips and tricks you can imagine...
Related Readings:
HELP WITH DATES (1997)
Calculating multiple shipping... (1997)
Artwork (1997)
Extended [ConvertChars] (1997)
WebCat & WebTen (1997)
Adding Shipping Costs based on percent of subtotal (1997)
Newbie problem blah blah blah (1997)
Erotic Sites (1997)
Multi Actions (1999)
2nd WebCatalog2 Feature Request (1996)
syntax question, not in online refernce (1997)
[OT] Thinking about a new Mac server (2004)
[FEATURE REQUEST] (2004)
test (2004)
how to know the [LineIndex] number webcat generated during a [AddLineItem]? (2000)
Another bug to squash (WebCat2b13 Mac .acgi) (1997)
Search design (1997)
off topic - dna snipets (1997)
Paymethod=cc allows all zeros - Anyone?? (1999)
[WebDNA] Auto Expansion of .zip (2009)