Re: [WebDNA] CORRECTION: 60% failure rate using replace in a loop

This WebDNA talk-list message is from

2010


It keeps the original formatting.
numero = 105622
interpreted = N
texte = Ken, something else: do you use your code locally? on a remote server? = using WebDNA.fcgi 7.0? with accelerator activated? i am doing some = basic tests on an accelerated WebDNA.fcgi 7.0 (local) and so far, i can = append 4,500 records per second and replacing 50,000 records takes less = than a second. Here my results Open a database (2 fields), append 50000 records using a loop 00:00:13=20= Replaces 50000 records 00:00:00=20 ReplaceFoundItems 50000 records 00:00:00=20 Just delete the records 00:00:00=20 global time: 00:00:13 Here the test code: ---------------------------------------------------------------------- [!] ##Check for the Database first - create it if not there--[/!] [showif T=3D[fileinfo file=3Dspeed.txt][exists][/fileinfo]] [closedatabase db=3Dspeed.txt] [deletefile file=3Dspeed.txt] [/showif] [showif F=3D[fileinfo file=3Dspeed.txt][exists][/fileinfo]] [writefile file=3Dspeed.txt]skuvalue [/writefile] [/showif] [!] ##Delete the records in the DB to ensure the search is 'clean'= --[/!] [delete db=3Dspeed.txt&neSKUdata=3Dfind_all] [text]start_time_global=3D[time][/text]
Open a database (2 fields), append 50000 records using a loop [Math time]{[time]}-{[start_time]}[/Math]
Replaces 50000 records [Math time]{[time]}-{[start_time]}[/Math]
ReplaceFoundItems 50000 records [Math time]{[time]}-{[start_time]}[/Math]
Just delete the records [Math time]{[time]}-{[start_time]}[/Math]
global time: [Math time]{[time]}-{[start_time_global]}[/Math] ---------------------------------------------------------------------- - chris On Jul 19, 2010, at 9:25, Kenneth Grome wrote: > Here's the code in my trigger template which is requested once per = second. Instead of trying to create 45 new records in a couple of = seconds I have resorted to creating only 1-4 new records per second = (although I want to create many more for a truly realistic auction = simulation). >=20 > The tickets.db holds all the bidderID (sku) values. I need to search = this db and retrieve a series of random bidderID's for the replace = contexts: >=20 >=20 > *************************************************************** >=20 > > [hideif [getchars start=3D4&end=3D5][time][/getchars]\2] > [showif [getchars start=3D7&end=3D8][time][/getchars]\11] > [search = db=3Dtest/tickets.db&eqemaildatarq=3DXXX&raemailsort=3D1&max=3D[math]ceil(= (1+[random])/25)[/math]][founditems] > [replace = db=3Dtest/bids.db&eqbidderIDdatarq=3D[blank]&asidxsort=3D1&idxtype=3Dnum&m= ax=3D1]bidTime=3D[time]&bidderID=3D[sku][/replace] > [/founditems][/search] > [/showif] > [/hideif] >=20 > > [showif [getchars start=3D4&end=3D5][time][/getchars]\6] > [showif [getchars start=3D7&end=3D8][time][/getchars]\17] > [search = db=3Dtest/tickets.db&eqemaildatarq=3DXXX&raemailsort=3D1&max=3D[math]ceil(= (1+[random])/25)[/math]][founditems] > [replace = db=3Dtest/bids.db&eqbidderIDdatarq=3D[blank]&asidxsort=3D1&idxtype=3Dnum&m= ax=3D1]bidTime=3D[time]&bidderID=3D[sku][/replace] > [/founditems][/search] > [/showif] > [/showif] >=20 > *************************************************************** >=20 >=20 > Naturally I cannot use this technique with replacefounditems because = I'm searching in a different db than where the records need to be = replaced. But there may be another way to approach this problem. I = think this might work faster: >=20 > 1- Search the tickets.db for (example) 45 random sku values > 2. Stores these skus in an indexed table > 3- Search the bids.db for the next 45 records with blank bidderID = values > 4- Use replacefounditems to change these bidderID's based on lookups = of the index values in the table >=20 > I don't know if this will be faster or not, but it's probably worth a = try as soon as I have some free time again. >=20 > Sincerely, > Kenneth Grome >=20 >=20 >=20 > --------------------------------------------------------- > This message is sent to you because you are subscribed to > the mailing list . > To unsubscribe, E-mail to: > archives: http://mail.webdna.us/list/talk@webdna.us > Bug Reporting: support@webdna.us Associated Messages, from the most recent to the oldest:

    
  1. Re: [WebDNA] CORRECTION: 60% failure rate using replace in a loop (christophe.billiottet@webdna.us 2010)
  2. Re: [WebDNA] CORRECTION: 60% failure rate using replace in a loop (Kenneth Grome 2010)
  3. Re: [WebDNA] CORRECTION: 60% failure rate using replace in a loop (Kenneth Grome 2010)
  4. Re: [WebDNA] CORRECTION: 60% failure rate using replace in a loop (christophe.billiottet@webdna.us 2010)
  5. Re: [WebDNA] CORRECTION: 60% failure rate using replace in a loop (Kenneth Grome 2010)
  6. Re: [WebDNA] CORRECTION: 60% failure rate using replace in a loop (Kenneth Grome 2010)
  7. Re: [WebDNA] CORRECTION: 60% failure rate using replace in a loop (christophe.billiottet@webdna.us 2010)
  8. Re: [WebDNA] CORRECTION: 60% failure rate using replace in a loop (Kenneth Grome 2010)
  9. Re: [WebDNA] CORRECTION: 60% failure rate using replace in a loop (christophe.billiottet@webdna.us 2010)
  10. Re: [WebDNA] CORRECTION: 60% failure rate using replace in a loop (Kenneth Grome 2010)
  11. Re: [WebDNA] CORRECTION: 60% failure rate using replace in a loop (christophe.billiottet@webdna.us 2010)
  12. Re: [WebDNA] CORRECTION: 60% failure rate using replace in a loop (Kenneth Grome 2010)
  13. Re: [WebDNA] CORRECTION: 60% failure rate using replace in a loop (christophe.billiottet@webdna.us 2010)
  14. [WebDNA] CORRECTION: 60% failure rate using replace in a loop (Kenneth Grome 2010)
Ken, something else: do you use your code locally? on a remote server? = using WebDNA.fcgi 7.0? with accelerator activated? i am doing some = basic tests on an accelerated WebDNA.fcgi 7.0 (local) and so far, i can = append 4,500 records per second and replacing 50,000 records takes less = than a second. Here my results Open a database (2 fields), append 50000 records using a loop 00:00:13=20= Replaces 50000 records 00:00:00=20 ReplaceFoundItems 50000 records 00:00:00=20 Just delete the records 00:00:00=20 global time: 00:00:13 Here the test code: ---------------------------------------------------------------------- [!] ##Check for the Database first - create it if not there--[/!] [showif T=3D[fileinfo file=3Dspeed.txt][exists][/fileinfo]] [closedatabase db=3Dspeed.txt] [deletefile file=3Dspeed.txt] [/showif] [showif F=3D[fileinfo file=3Dspeed.txt][exists][/fileinfo]] [writefile file=3Dspeed.txt]skuvalue [/writefile] [/showif] [!] ##Delete the records in the DB to ensure the search is 'clean'= --[/!] [delete db=3Dspeed.txt&neSKUdata=3Dfind_all] [text]start_time_global=3D[time][/text]
Open a database (2 fields), append 50000 records using a loop [Math time]{[time]}-{[start_time]}[/Math]
Replaces 50000 records [Math time]{[time]}-{[start_time]}[/Math]
ReplaceFoundItems 50000 records [Math time]{[time]}-{[start_time]}[/Math]
Just delete the records [Math time]{[time]}-{[start_time]}[/Math]
global time: [Math time]{[time]}-{[start_time_global]}[/Math] ---------------------------------------------------------------------- - chris On Jul 19, 2010, at 9:25, Kenneth Grome wrote: > Here's the code in my trigger template which is requested once per = second. Instead of trying to create 45 new records in a couple of = seconds I have resorted to creating only 1-4 new records per second = (although I want to create many more for a truly realistic auction = simulation). >=20 > The tickets.db holds all the bidderID (sku) values. I need to search = this db and retrieve a series of random bidderID's for the replace = contexts: >=20 >=20 > *************************************************************** >=20 > > [hideif [getchars start=3D4&end=3D5][time][/getchars]\2] > [showif [getchars start=3D7&end=3D8][time][/getchars]\11] > [search = db=3Dtest/tickets.db&eqemaildatarq=3DXXX&raemailsort=3D1&max=3D[math]ceil(= (1+[random])/25)[/math]][founditems] > [replace = db=3Dtest/bids.db&eqbidderIDdatarq=3D[blank]&asidxsort=3D1&idxtype=3Dnum&m= ax=3D1]bidTime=3D[time]&bidderID=3D[sku][/replace] > [/founditems][/search] > [/showif] > [/hideif] >=20 > > [showif [getchars start=3D4&end=3D5][time][/getchars]\6] > [showif [getchars start=3D7&end=3D8][time][/getchars]\17] > [search = db=3Dtest/tickets.db&eqemaildatarq=3DXXX&raemailsort=3D1&max=3D[math]ceil(= (1+[random])/25)[/math]][founditems] > [replace = db=3Dtest/bids.db&eqbidderIDdatarq=3D[blank]&asidxsort=3D1&idxtype=3Dnum&m= ax=3D1]bidTime=3D[time]&bidderID=3D[sku][/replace] > [/founditems][/search] > [/showif] > [/showif] >=20 > *************************************************************** >=20 >=20 > Naturally I cannot use this technique with replacefounditems because = I'm searching in a different db than where the records need to be = replaced. But there may be another way to approach this problem. I = think this might work faster: >=20 > 1- Search the tickets.db for (example) 45 random sku values > 2. Stores these skus in an indexed table > 3- Search the bids.db for the next 45 records with blank bidderID = values > 4- Use replacefounditems to change these bidderID's based on lookups = of the index values in the table >=20 > I don't know if this will be faster or not, but it's probably worth a = try as soon as I have some free time again. >=20 > Sincerely, > Kenneth Grome >=20 >=20 >=20 > --------------------------------------------------------- > This message is sent to you because you are subscribed to > the mailing list . > To unsubscribe, E-mail to: > archives: http://mail.webdna.us/list/talk@webdna.us > Bug Reporting: support@webdna.us christophe.billiottet@webdna.us

DOWNLOAD WEBDNA NOW!

Top Articles:

Talk List

The WebDNA community talk-list is the best place to get some help: several hundred extremely proficient programmers with an excellent knowledge of WebDNA and an excellent spirit will deliver all the tips and tricks you can imagine...

Related Readings:

European Convention (2004) WebDNA CAPTCHA (2008) Using Plug-In while running 1.6.1 (1997) PSC recommends what date format yr 2000??? (1997) calculating tax rates, mail order solutions and version 2 (1997) Nested Loops and SHOWIFs (1997) AddLineItem Problem (1997) Keep away (1997) Simple and Direct Question for SMSI (2002) OSX (2002) Problems with [Search] param - Mac Plugin b15 (1997) NT vs Mac (1997) Setting up WebCatalog with Retail Pro data (1996) WebCatalog for Postcards ? (1997) Append File help needed (2000) WebCat 4 / 4.5 License for sale! (2003) New commands in Final candidate (1997) Sorting Numbers (1997) Email within tmpl ? (1997) question: webdelivery folder and file (1997)