Re: Slow Code, how to speed up
This WebDNA talk-list message is from 2002
It keeps the original formatting.
numero = 41119
interpreted = N
texte = >Here's code that takes 1600 ticks to process on my server. How can I speed it up? It is opening a 700k file and then processing it, and the file is getting larger at all times, since it is a list of referrering web sites. > >[!] Search Temp Database [/!] >[search db=referer.txt&neREFERRERdatarq=findall&REFERRERsumm=t][founditems] > >[!] Append to Temp 2 Database [/!] >[append db=templog2.db]referrer=[referrer]&count=[search >db=referer.txt&eqREFERRERdatarq=[url][REFERRER][/url]&deDTsort=1&DTtype=date >&max=1][numfound]&dt=[founditems][dt][/founditems][/search][/append] >[/founditems][/search]That inner [append] is taking more time because it is opening and closing the database file once per time through your outer [founditems] loop, and all that file access is killing the speed.Instead, open templog2.db just once outside the loop, then write raw data to the end of templog2.db using [appendfile], and then flush templog2.db to force it to be read back in as a database. Here's pseudo-code for it. You have to be *very* careful about your linebreaks here, because you don't want extra carriage returns in the middle of your templog.db file to screw up the db format:close templog2.db [text]tab=%09[/text] [appendfile file=templog2.db][search db=referer.txt&blah][founditems][referrer][tab][your count stuff][tab][your dt stuff] [/founditems][/appendfile]The idea here is that with [appendfile] you are just opening the file once, and doing all the writing to it inside your loop. -- ______________________________________________________________________________ Grant Hulbert | Internet Software Inventor | http://www.hulbertfamily.com/------------------------------------------------------------- This message is sent to you because you are subscribed to the mailing list
. To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to Web Archive of this list is at: http://search.smithmicro.com/
Associated Messages, from the most recent to the oldest:
>Here's code that takes 1600 ticks to process on my server. How can I speed it up? It is opening a 700k file and then processing it, and the file is getting larger at all times, since it is a list of referrering web sites. > >[!] Search Temp Database [/!] >[search db=referer.txt&neREFERRERdatarq=findall&REFERRERsumm=t][founditems] > >[!] Append to Temp 2 Database [/!] >[append db=templog2.db]referrer=[referrer]&count=[search >db=referer.txt&eqREFERRERdatarq=[url][referrer][/url]&deDTsort=1&DTtype=date >&max=1][numfound]&dt=[founditems][dt][/founditems][/search][/append] >[/founditems][/search]That inner [append] is taking more time because it is opening and closing the database file once per time through your outer [founditems] loop, and all that file access is killing the speed.Instead, open templog2.db just once outside the loop, then write raw data to the end of templog2.db using [appendfile], and then flush templog2.db to force it to be read back in as a database. Here's pseudo-code for it. You have to be *very* careful about your linebreaks here, because you don't want extra carriage returns in the middle of your templog.db file to screw up the db format:close templog2.db [text]tab=%09[/text] [appendfile file=templog2.db][search db=referer.txt&blah][founditems][referrer][tab][your count stuff][tab][your dt stuff] [/founditems][/appendfile]The idea here is that with [appendfile] you are just opening the file once, and doing all the writing to it inside your loop. -- ______________________________________________________________________________ Grant Hulbert | Internet Software Inventor | http://www.hulbertfamily.com/------------------------------------------------------------- This message is sent to you because you are subscribed to the mailing list . To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to Web Archive of this list is at: http://search.smithmicro.com/
Grant Hulbert
DOWNLOAD WEBDNA NOW!
Top Articles:
Talk List
The WebDNA community talk-list is the best place to get some help: several hundred extremely proficient programmers with an excellent knowledge of WebDNA and an excellent spirit will deliver all the tips and tricks you can imagine...
Related Readings:
Help DebugLevel Content-type: text/html (1998)
Quit revisited (1997)
# of real domains on 1 web server (1997)
WebCat2b15MacPlugin - showing [math] (1997)
WC2.0 Memory Requirements (1997)
emailer truncates last letter! (1997)
WebCat2 - storing unformatted date data? (1997)
bug? template caching and apache transfer byte log (1999)
NT [delete] (1998)
WebCat2b13MacPlugIn - [showif][search][/showif] (1997)
OT - Useful Shipping Info (2002)
FW: 2.01 upgrade problems (1997)
Emailer help....! (1997)
[WebDNA] [append] does not add the newline char at the end of (2009)
R.I.P. Netscape (2003)
Price and Formula.db (2002)
Ok here is a question? (1997)
docs for WebCatalog2 (1997)
Plugin or CGI or both (1997)
WebCat2 Append problem (B14Macacgi) (1997)