Re: Multithreading of [replace]
This WebDNA talk-list message is from 1999
It keeps the original formatting.
numero = 23096
interpreted = N
texte = >Given that I have written a database to do logging of visitors to the website, >so that I can do my own analysis of who did what, I am running into a problem >with the sheer number of write commands being issued to this database>[search db=carts.db&eqCARTdata=[cart]&DATEtype=date&DATEsort=1&DATEsdir=de] >[showif [numfound]=0] >[Append db=carts.db] >cart=[cart]&date=[date]&firstpage=[thisurl]&lastpage=[thisurl] >[/append] >[/showif] >[showif [numfound]=1] >[replace db=carts.db&eqCARTdata=[cart]] >lastpage=[thisurl] >[/replace] >[/showif] >[/search]>So, Grant, here is my question: Is there anything that can be optimized that >will alleviate the speed hit this code causes under load?Multithreading is somewhat slowed here, because carts.db is locked up during the replace, so no other threads can even read it (other incoming hits get 'stuck' at the first search until the replace is done). What OS is this? For multithreading NT would be fastest, MacOS plugin next, and finally MacOS ACGI. Play with the prefs value of ThreadSwitchTicks; making it smaller smooths out multithreading but can introduce extra overhead as well, so it needs tuning.Date searching can be very slow compared to plain text, because WebCatalog must interpret those dates and calculate lots of stuff. I would re-order your date text into YYYY/MM/DD order so that it is text-sortable *without* needing to specify DATEtype=date. I would also set allCase=T so searches and replaces are case-sensitive (a bit faster).Finally, you can get rid of that whole ShowIf thing: unless I misunderstand your intention, we already put a feature into replace that will append if not found: [replace db=xx&append=T] will append a new record if none is found.I hope this helps! I think doing all of these things could net you from 50% to 500% speed increase.Grant Hulbert, V.P. Engineering | ==== eCommerce for the Rest of Us ==== Pacific Coast Software | WebCatalog, WebMerchant, 11770 Bernardo Plaza Court | SiteEdit Pro, PhotoMaster, San Diego, CA 92128 | Typhoon 619/675-1106 Fax: 619/675-0372 | http://www.smithmicro.com/
Associated Messages, from the most recent to the oldest:
>Given that I have written a database to do logging of visitors to the website, >so that I can do my own analysis of who did what, I am running into a problem >with the sheer number of write commands being issued to this database>[search db=carts.db&eqCARTdata=
[cart]&DATEtype=date&DATEsort=1&DATEsdir=de] >[showif [numfound]=0] >[Append db=carts.db] >cart=
[cart]&date=
[date]&firstpage=
[thisurl]&lastpage=
[thisurl] >[/append] >[/showif] >[showif [numfound]=1] >[replace db=carts.db&eqCARTdata=
[cart]] >lastpage=
[thisurl] >[/replace] >[/showif] >[/search]>So, Grant, here is my question: Is there anything that can be optimized that >will alleviate the speed hit this code causes under load?Multithreading is somewhat slowed here, because carts.db is locked up during the replace, so no other threads can even read it (other incoming hits get 'stuck' at the first search until the replace is done). What OS is this? For multithreading NT would be fastest, MacOS plugin next, and finally MacOS ACGI. Play with the prefs value of ThreadSwitchTicks; making it smaller smooths out multithreading but can introduce extra overhead as well, so it needs tuning.Date searching can be very slow compared to plain text, because WebCatalog must interpret those dates and calculate lots of stuff. I would re-order your date text into YYYY/MM/DD order so that it is text-sortable *without* needing to specify DATEtype=date. I would also set allCase=T so searches and replaces are case-sensitive (a bit faster).Finally, you can get rid of that whole ShowIf thing: unless I misunderstand your intention, we already put a feature into replace that will append if not found: [replace db=xx&append=T] will append a new record if none is found.I hope this helps! I think doing all of these things could net you from 50% to 500% speed increase.Grant Hulbert, V.P. Engineering | ==== eCommerce for the Rest of Us ==== Pacific Coast Software | WebCatalog, WebMerchant, 11770 Bernardo Plaza Court | SiteEdit Pro, PhotoMaster, San Diego, CA 92128 | Typhoon 619/675-1106 Fax: 619/675-0372 | http://www.smithmicro.com/
Grant Hulbert
DOWNLOAD WEBDNA NOW!
Top Articles:
Talk List
The WebDNA community talk-list is the best place to get some help: several hundred extremely proficient programmers with an excellent knowledge of WebDNA and an excellent spirit will deliver all the tips and tricks you can imagine...
Related Readings:
Permissions (2000)
delete a file? (1998)
[WebDNA] Cannot get WebDNA and MySQL working. (2015)
Re:Running 2 two WebCatalog.acgi's (1996)
[ADDLINEITEM] hangs Web* (1998)
Searchable WebCat (etc.) Docs ? (1997)
quotes and truncating? (1997)
[cart] number range ?? (2001)
RequiredFields notes ... (1997)
FW: Shipping calculations (1997)
Re:Running 2 two WebCatalog.acgi's (1996)
is sku a REQUIRED field on NT (1997)
Calendar (1997)
How to delete all files in a folder? (2001)
RAM variables (1997)
Customer - again (1998)
What am I missing (1997)
WC2b12: Yes, Formulas.db is for real (1997)
Dates - Search and Sorting (1999)
WebCatalog seems to choke on large (2meg) html files. (1998)