Re: [WebDNA] Append speed limits
This WebDNA talk-list message is from 2008
It keeps the original formatting.
numero = 101428
interpreted = N
texte = 20,000 records per second is a lot. I think the RAM-based application is necessary (disk-based is by way too slow) but am not sure a quad processor would properly distribute the load over the CPUs. If the hits you expect come from several nodes, what is would do is a round robin between several WebDNA servers using a network load balancing system (Alteon, f5-networks..), then write to disk afterwards, then merge the data in a single file. Not easy anyway.... - chris On Nov 15, 2008, at 1:25, Kenneth Grome wrote: > I've considered it but the business model requires that the > data itself is not maintianed by cloud services. > > Google has something similar BTW, I think they call it "Big > Table". When I read about these options the first thing > that occurred to me is how similar they may be to webdna's > internal structure. > > I don't know webdna's internal structure of course so I'm > just guessing, but from what I've learned in my own > experience, and also from the things Grant has explained > over the years, I think there are more than a few > similarities. > > Sincerely, > Ken Grome > > > > > > >> If the dataset gets too large for RAM based db >> management, you could try >> http://aws.amazon.com/simpledb/. > --------------------------------------------------------- > This message is sent to you because you are subscribed to > the mailing list
. > To unsubscribe, E-mail to: > archives: http://mail.webdna.us/list/talk@webdna.us > old archives: http://dev.webdna.us/TalkListArchive/
Associated Messages, from the most recent to the oldest:
20,000 records per second is a lot. I think the RAM-based application is necessary (disk-based is by way too slow) but am not sure a quad processor would properly distribute the load over the CPUs. If the hits you expect come from several nodes, what is would do is a round robin between several WebDNA servers using a network load balancing system (Alteon, f5-networks..), then write to disk afterwards, then merge the data in a single file. Not easy anyway.... - chris On Nov 15, 2008, at 1:25, Kenneth Grome wrote: > I've considered it but the business model requires that the > data itself is not maintianed by cloud services. > > Google has something similar BTW, I think they call it "Big > Table". When I read about these options the first thing > that occurred to me is how similar they may be to webdna's > internal structure. > > I don't know webdna's internal structure of course so I'm > just guessing, but from what I've learned in my own > experience, and also from the things Grant has explained > over the years, I think there are more than a few > similarities. > > Sincerely, > Ken Grome > > > > > > >> If the dataset gets too large for RAM based db >> management, you could try >> http://aws.amazon.com/simpledb/. > --------------------------------------------------------- > This message is sent to you because you are subscribed to > the mailing list . > To unsubscribe, E-mail to: > archives: http://mail.webdna.us/list/talk@webdna.us > old archives: http://dev.webdna.us/TalkListArchive/
christophe.billiottet@webdna.us
DOWNLOAD WEBDNA NOW!
Top Articles:
Talk List
The WebDNA community talk-list is the best place to get some help: several hundred extremely proficient programmers with an excellent knowledge of WebDNA and an excellent spirit will deliver all the tips and tricks you can imagine...
Related Readings:
WebCat2 - [include] tags (1997)
Re (2003)
Replace Statement (1997)
Search context not finding recent entries (1998)
Can anyone help me with this shipping thing? (1998)
Re2: frames & carts (1997)
Server Traffic Simulation (1998)
Quit revisited (1997)
TCPConnect & TCPSend (2003)
security (1997)
test (2002)
Multiple CPUs? (2000)
WebCat editing, SiteGuard & SiteEdit (1997)
Unauthorized User trouble (1998)
Running 2 two WebCatalog.acgi's (1996)
shipcost (1997)
Appending Information from one db to another (1998)
Re[2]: Price Not Appearing (2000)
Nested tags count question (1997)
[WebDNA] Show next..... (2010)