Re: [WebDNA] HTTP Streaming - POSSIBLE!
This WebDNA talk-list message is from 2010
It keeps the original formatting.
numero = 105581
interpreted = N
texte = How about upon the creation of a new sequence file it deletes the old. Then it knows what file to wait for because you base the file name of numfound+1 search of auction bids. So, client arrives, gets the number of current bids and then it knows what file to wait on. Shooting in the dark, just another idea... Scott On Jul 13, 2010, at 8:16 AM, Kenneth Grome wrote: >> Let's say a folder is created for each auction item. > > There is only one auction item so this part is easy ... :) > > >> Then every time a new bid comes in it writes a >> new sequential file. > > We will have anywhere from 1000 to 5000 bidders submitting bids > during the one-hour auction. My estimate of the average number of > bids received per second is 55. This means 55 new files written per > second "on average" ... > > But this is only an average. During the early part of the auction > only one or two bids may come in each second, or perhaps only one or > two bids each minute. It's when we're nearing the end of the > auction that I expect to receive thousands of bids per second. > > >> Now when a new bidder views a potential item it >> would search the number of files in that auctions >> directory and would keep the connection open with a >> waitforfile command until a new bid comes in. >> Then when the file is created it simply refreshes the >> page and waits for the next bid sequential file. > > My concern here is that there will be tens of thousands of "bid > files" created during the one-hour auction, and maybe webdna won't > be able to count them fast enough to respond quickly, especially > near the end of the auction when thousands of bidders are online and > submitting new bids rapidly. > > >> This solves your server load issues because it doesn't >> demand packets to constantly be exchanged. > > Yes, it solves the network part of the server load issue. But it > burdens webdna with a task that invokes the OS to count files on > disk -- and write new files in rapid succession -- instead of > keeping all this data in a ram-cached webdna database. Because of > all the disk access inherent in counting files on disk and writing > new files all the time, I'm not convinced that webdna will keep up. > > Webdna is fast when writing to its own ram-cached databases since > this is all done in memory. If this data is not flushed to disk > until *after* the auction there would be very few disk disk hits > during the auction. > > But my gut tells me that writing every new bid to a separate file on > disk, and then invoking the OS to count those files thousands of > times per second (one count for each of the thousands of bidders who > are online and waiting for the next file to be written) is just > going to bog down the server and fail miserably. > > I could be wrong of course, but I have enough doubts about this > type of system to avoid proceeding with it -- at least until someone > comes up with an argument that convinces me otherwise. > > Sincerely, > Kenneth Grome > > > > --------------------------------------------------------- > This message is sent to you because you are subscribed to > the mailing list
. > To unsubscribe, E-mail to: > archives: http://mail.webdna.us/list/talk@webdna.us > Bug Reporting: support@webdna.us >
Associated Messages, from the most recent to the oldest:
How about upon the creation of a new sequence file it deletes the old. Then it knows what file to wait for because you base the file name of numfound+1 search of auction bids. So, client arrives, gets the number of current bids and then it knows what file to wait on. Shooting in the dark, just another idea... Scott On Jul 13, 2010, at 8:16 AM, Kenneth Grome wrote: >> Let's say a folder is created for each auction item. > > There is only one auction item so this part is easy ... :) > > >> Then every time a new bid comes in it writes a >> new sequential file. > > We will have anywhere from 1000 to 5000 bidders submitting bids > during the one-hour auction. My estimate of the average number of > bids received per second is 55. This means 55 new files written per > second "on average" ... > > But this is only an average. During the early part of the auction > only one or two bids may come in each second, or perhaps only one or > two bids each minute. It's when we're nearing the end of the > auction that I expect to receive thousands of bids per second. > > >> Now when a new bidder views a potential item it >> would search the number of files in that auctions >> directory and would keep the connection open with a >> waitforfile command until a new bid comes in. >> Then when the file is created it simply refreshes the >> page and waits for the next bid sequential file. > > My concern here is that there will be tens of thousands of "bid > files" created during the one-hour auction, and maybe webdna won't > be able to count them fast enough to respond quickly, especially > near the end of the auction when thousands of bidders are online and > submitting new bids rapidly. > > >> This solves your server load issues because it doesn't >> demand packets to constantly be exchanged. > > Yes, it solves the network part of the server load issue. But it > burdens webdna with a task that invokes the OS to count files on > disk -- and write new files in rapid succession -- instead of > keeping all this data in a ram-cached webdna database. Because of > all the disk access inherent in counting files on disk and writing > new files all the time, I'm not convinced that webdna will keep up. > > Webdna is fast when writing to its own ram-cached databases since > this is all done in memory. If this data is not flushed to disk > until *after* the auction there would be very few disk disk hits > during the auction. > > But my gut tells me that writing every new bid to a separate file on > disk, and then invoking the OS to count those files thousands of > times per second (one count for each of the thousands of bidders who > are online and waiting for the next file to be written) is just > going to bog down the server and fail miserably. > > I could be wrong of course, but I have enough doubts about this > type of system to avoid proceeding with it -- at least until someone > comes up with an argument that convinces me otherwise. > > Sincerely, > Kenneth Grome > > > > --------------------------------------------------------- > This message is sent to you because you are subscribed to > the mailing list . > To unsubscribe, E-mail to: > archives: http://mail.webdna.us/list/talk@webdna.us > Bug Reporting: support@webdna.us >
Scott Walters
DOWNLOAD WEBDNA NOW!
Top Articles:
Talk List
The WebDNA community talk-list is the best place to get some help: several hundred extremely proficient programmers with an excellent knowledge of WebDNA and an excellent spirit will deliver all the tips and tricks you can imagine...
Related Readings:
Adding up weights (1998)
Rhapsody? (1997)
WebCatalog Mac and cgi-bin (WebSTAR 2.0) (1997)
WebStar5 and realm authentication (2003)
How did *you* learn Web Catalog? (2000)
HELP WITH DATES (1997)
WebCat2 several catalogs? (1997)
Writefile doesn't work, which permissions to use? (2005)
ODBC Performance? (2001)
Re:2nd WebCatalog2 Feature Request (1996)
convert characters (2003)
RE: How much ram??? (1997)
Duplicates (1998)
pretty Urgent: Crash after addlineitem (2001)
hierarchy question... (2000)
[WebDNA] Encode cookies ONLY via "method=Base64" (2008)
WebCat2 beta FTP site (1997)
WebCat2 several catalogs? (1997)
Re:trouble (1997)
frames & carts (1997)