Re: Slow Code, how to speed up
This WebDNA talk-list message is from 2002
It keeps the original formatting.
numero = 41119
interpreted = N
texte = >Here's code that takes 1600 ticks to process on my server. How can I speed it up? It is opening a 700k file and then processing it, and the file is getting larger at all times, since it is a list of referrering web sites. > >[!] Search Temp Database [/!] >[search db=referer.txt&neREFERRERdatarq=findall&REFERRERsumm=t][founditems] > >[!] Append to Temp 2 Database [/!] >[append db=templog2.db]referrer=[referrer]&count=[search >db=referer.txt&eqREFERRERdatarq=[url][REFERRER][/url]&deDTsort=1&DTtype=date >&max=1][numfound]&dt=[founditems][dt][/founditems][/search][/append] >[/founditems][/search]That inner [append] is taking more time because it is opening and closing the database file once per time through your outer [founditems] loop, and all that file access is killing the speed.Instead, open templog2.db just once outside the loop, then write raw data to the end of templog2.db using [appendfile], and then flush templog2.db to force it to be read back in as a database. Here's pseudo-code for it. You have to be *very* careful about your linebreaks here, because you don't want extra carriage returns in the middle of your templog.db file to screw up the db format:close templog2.db [text]tab=%09[/text] [appendfile file=templog2.db][search db=referer.txt&blah][founditems][referrer][tab][your count stuff][tab][your dt stuff] [/founditems][/appendfile]The idea here is that with [appendfile] you are just opening the file once, and doing all the writing to it inside your loop. -- ______________________________________________________________________________ Grant Hulbert | Internet Software Inventor | http://www.hulbertfamily.com/------------------------------------------------------------- This message is sent to you because you are subscribed to the mailing list
. To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to Web Archive of this list is at: http://search.smithmicro.com/
Associated Messages, from the most recent to the oldest:
>Here's code that takes 1600 ticks to process on my server. How can I speed it up? It is opening a 700k file and then processing it, and the file is getting larger at all times, since it is a list of referrering web sites. > >[!] Search Temp Database [/!] >[search db=referer.txt&neREFERRERdatarq=findall&REFERRERsumm=t][founditems] > >[!] Append to Temp 2 Database [/!] >[append db=templog2.db]referrer=[referrer]&count=[search >db=referer.txt&eqREFERRERdatarq=[url][referrer][/url]&deDTsort=1&DTtype=date >&max=1][numfound]&dt=[founditems][dt][/founditems][/search][/append] >[/founditems][/search]That inner [append] is taking more time because it is opening and closing the database file once per time through your outer [founditems] loop, and all that file access is killing the speed.Instead, open templog2.db just once outside the loop, then write raw data to the end of templog2.db using [appendfile], and then flush templog2.db to force it to be read back in as a database. Here's pseudo-code for it. You have to be *very* careful about your linebreaks here, because you don't want extra carriage returns in the middle of your templog.db file to screw up the db format:close templog2.db [text]tab=%09[/text] [appendfile file=templog2.db][search db=referer.txt&blah][founditems][referrer][tab][your count stuff][tab][your dt stuff] [/founditems][/appendfile]The idea here is that with [appendfile] you are just opening the file once, and doing all the writing to it inside your loop. -- ______________________________________________________________________________ Grant Hulbert | Internet Software Inventor | http://www.hulbertfamily.com/------------------------------------------------------------- This message is sent to you because you are subscribed to the mailing list . To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to Web Archive of this list is at: http://search.smithmicro.com/
Grant Hulbert
DOWNLOAD WEBDNA NOW!
Top Articles:
Talk List
The WebDNA community talk-list is the best place to get some help: several hundred extremely proficient programmers with an excellent knowledge of WebDNA and an excellent spirit will deliver all the tips and tricks you can imagine...
Related Readings:
WebCatb15 Mac CGI -- [purchase] (1997)
multi-paragraph fields (1997)
Word Breaks (1998)
Re1000001: Setting up shop (1997)
Secure Server (1997)
Web Catalog 2 demo (1997)
no global [username] or [password] displayed ... (1997)
Bulk Linebreak Changes (2002)
syntax question, not in online refernce (1997)
WebCat2b12 CGI Mac - [shownext] problem (1997)
Counting records (2000)
Nested tags count question (1997)
process SSI (1998)
What am I missing (1997)
I have observed and I am gone. (1998)
Re:Searching for ALL / empty form field (1997)
[append] vs. [appendfile] delta + question? (1997)
GuestBook example (1997)
select multiple 2 more cents (1997)
WebCat2 beta 11 - new prefs ... (1997)