Re: Multithreading of [replace]
This WebDNA talk-list message is from 1999
It keeps the original formatting.
numero = 23094
interpreted = N
texte = > Brian, > > What about another approach. How about using the following code instead: > > [appendfile logs/[date > %d%b%Y].txt][cart]
[date][thisurl]etc...[/appendfile] > > This way, every day you get a new logfile with all of your hits appended to > one large text file. Now every week, you can do a listFiles command and > sweep up these databases, summarize on the cart field (you need to create > .hdr files) to get unique records, the produce whatever reports you need. > > This shouldn't impact your server much as you are appending to an existing > file (first hit of day auto creates a new file). If you are logging a lot of > information that relies on logic, you may want to wrap this in a spawn > context. > > Aloha, Olin 3 reasons I don't do this. a) no visitor count on the fly. b) how do you figure first page and last page, start_visit_time and end_visit_time? c) how do you put this info into the shopping cart when the check out? d) BONUS - How do you read into ram the resulting 350 meg file,(assuming one lime per page per visitor, along with all of the other info captured)The database is the best solution, I just can't figure out how to make it work efficiently.If I spawn a process off, does it remember the variables of the page that created the spawn? (IE cart) Brian B. Burton BOFH - Department of Redundancy Department --------------------------------------------------------------- MMT Solutions - Specializing in Online Shopping Solutions 973-808-8644 http://www.safecommerce.comAre you a Web Programmer? I am today.> >> -----Original Message----- >> From: Brian B. Burton [mailto:brian@burtons.com] >> Sent: Friday, January 29, 1999 2:38 PM >> To: WebDNA-Talk >> Subject: Multithreading of [replace] >> >> >> This question is mostly for Grant H. as I think he probably would >> be the most >> qualified person to answer this. >> >> Given that I have written a database to do logging of visitors to >> the website, >> so that I can do my own analysis of who did what, I am running >> into a problem >> with the sheer number of write commands being issued to this >> database, and it >> affecting the speed of the whole website. The code shown below is >> at the top of >> every webpage. (the carriage returns are mine, for this email, >> and are not on >> the actual code; also the info appended and replaced is reduced >> for this email, >> in reality, almost 20 fields are set) >> >> [search >> db=carts.db&eqCARTdata=[cart]&DATEtype=date&DATEsort=1&DATEsdir=de] >> [showif [numfound]=0] >> [Append db=carts.db] >> cart=[cart]&date=[date]&firstpage=[thisurl]&lastpage=[thisurl] >> [/append] >> [/showif] >> [showif [numfound]=1] >> [replace db=carts.db&eqCARTdata=[cart]] >> lastpage=[thisurl] >> [/replace] >> [/showif] >> [/search] >> >> >> Now, I am using a search instead of two lookups, but I am under >> the impression >> that searching records isn't very time consuming. This code works >> just fine >> until there are about 25-30 connections simultaneously on the >> server, at which >> point this code almost locks the poor server up. Please remember, >> this is at the >> top of each and every page viewed. I can only assume that one >> page can not load >> until the last finishes, because this code causes a database >> write thus locking >> read access to the database until the write is complete. Also, >> with about 7000 >> records in the database (two days worth for one site) even at >> lower connections >> (10+) this runs kind of slow. I'm sure the server would be plenty >> snappy if I >> didn't have this on the pages, but unfortunately, the only other >> option is to >> write it into cart headers, causing the creation of a file (5000 >> a day) for >> every cart issued. >> >> So, Grant, here is my question: Is there anything that can be >> optimized that >> will alleviate the speed hit this code causes under load?
Associated Messages, from the most recent to the oldest:
> Brian, > > What about another approach. How about using the following code instead: > > [appendfile logs/[date > %d%b%Y].txt][cart][date][thisurl]etc...[/appendfile] > > This way, every day you get a new logfile with all of your hits appended to > one large text file. Now every week, you can do a listFiles command and > sweep up these databases, summarize on the cart field (you need to create > .hdr files) to get unique records, the produce whatever reports you need. > > This shouldn't impact your server much as you are appending to an existing > file (first hit of day auto creates a new file). If you are logging a lot of > information that relies on logic, you may want to wrap this in a spawn > context. > > Aloha, Olin 3 reasons I don't do this. a) no visitor count on the fly. b) how do you figure first page and last page, start_visit_time and end_visit_time? c) how do you put this info into the shopping cart when the check out? d) BONUS - How do you read into ram the resulting 350 meg file,(assuming one lime per page per visitor, along with all of the other info captured)The database is the best solution, I just can't figure out how to make it work efficiently.If I spawn a process off, does it remember the variables of the page that created the spawn? (IE cart) Brian B. Burton BOFH - Department of Redundancy Department --------------------------------------------------------------- MMT Solutions - Specializing in Online Shopping Solutions 973-808-8644 http://www.safecommerce.comAre you a Web Programmer? I am today.> >> -----Original Message----- >> From: Brian B. Burton [mailto:brian@burtons.com] >> Sent: Friday, January 29, 1999 2:38 PM >> To: WebDNA-Talk >> Subject: Multithreading of [replace] >> >> >> This question is mostly for Grant H. as I think he probably would >> be the most >> qualified person to answer this. >> >> Given that I have written a database to do logging of visitors to >> the website, >> so that I can do my own analysis of who did what, I am running >> into a problem >> with the sheer number of write commands being issued to this >> database, and it >> affecting the speed of the whole website. The code shown below is >> at the top of >> every webpage. (the carriage returns are mine, for this email, >> and are not on >> the actual code; also the info appended and replaced is reduced >> for this email, >> in reality, almost 20 fields are set) >> >> [search >> db=carts.db&eqCARTdata=[cart]&DATEtype=date&DATEsort=1&DATEsdir=de] >> [showif [numfound]=0] >> [Append db=carts.db] >> cart=[cart]&date=[date]&firstpage=[thisurl]&lastpage=[thisurl] >> [/append] >> [/showif] >> [showif [numfound]=1] >> [replace db=carts.db&eqCARTdata=[cart]] >> lastpage=[thisurl] >> [/replace] >> [/showif] >> [/search] >> >> >> Now, I am using a search instead of two lookups, but I am under >> the impression >> that searching records isn't very time consuming. This code works >> just fine >> until there are about 25-30 connections simultaneously on the >> server, at which >> point this code almost locks the poor server up. Please remember, >> this is at the >> top of each and every page viewed. I can only assume that one >> page can not load >> until the last finishes, because this code causes a database >> write thus locking >> read access to the database until the write is complete. Also, >> with about 7000 >> records in the database (two days worth for one site) even at >> lower connections >> (10+) this runs kind of slow. I'm sure the server would be plenty >> snappy if I >> didn't have this on the pages, but unfortunately, the only other >> option is to >> write it into cart headers, causing the creation of a file (5000 >> a day) for >> every cart issued. >> >> So, Grant, here is my question: Is there anything that can be >> optimized that >> will alleviate the speed hit this code causes under load?
Brian B. Burton
DOWNLOAD WEBDNA NOW!
Top Articles:
Talk List
The WebDNA community talk-list is the best place to get some help: several hundred extremely proficient programmers with an excellent knowledge of WebDNA and an excellent spirit will deliver all the tips and tricks you can imagine...
Related Readings:
[WRITEFILE] to create database spider food? (1998)
Paths, relative paths, webstar server setup and security (1997)
Re:Signal Raised (1997)
Re2: Calculating multiple shipping... (1997)
RE: Languages (1997)
FEA REQ: One .hdr, multiple .db's (2003)
RE: what characters are replaced for tab and CR? (1998)
Navigator 4.01 (1997)
taxTotal, grandTotal (1997)
webcat (2000)
Re:Virtual hosting and webcatNT (1997)
Setting up WebCatalog with Retail Pro data (1996)
MOOOOOO (2000)
PC Authorize & Webmerchant (1998)
RE: WebDNA-Talk searchable? (1997)
WCS Newbie question (1997)
Accepting credit cards (1997)
Help! WebCat2 bug (1997)
Emailer choke (1997)
WebCat2: Items xx to xx shown, etc. (1997)