Re: [WebDNA] CORRECTION: 60% failure rate using replace in a loop

This WebDNA talk-list message is from

2010


It keeps the original formatting.
numero = 105622
interpreted = N
texte = Ken, something else: do you use your code locally? on a remote server? = using WebDNA.fcgi 7.0? with accelerator activated? i am doing some = basic tests on an accelerated WebDNA.fcgi 7.0 (local) and so far, i can = append 4,500 records per second and replacing 50,000 records takes less = than a second. Here my results Open a database (2 fields), append 50000 records using a loop 00:00:13=20= Replaces 50000 records 00:00:00=20 ReplaceFoundItems 50000 records 00:00:00=20 Just delete the records 00:00:00=20 global time: 00:00:13 Here the test code: ---------------------------------------------------------------------- [!] ##Check for the Database first - create it if not there--[/!] [showif T=3D[fileinfo file=3Dspeed.txt][exists][/fileinfo]] [closedatabase db=3Dspeed.txt] [deletefile file=3Dspeed.txt] [/showif] [showif F=3D[fileinfo file=3Dspeed.txt][exists][/fileinfo]] [writefile file=3Dspeed.txt]skuvalue [/writefile] [/showif] [!] ##Delete the records in the DB to ensure the search is 'clean'= --[/!] [delete db=3Dspeed.txt&neSKUdata=3Dfind_all] [text]start_time_global=3D[time][/text]
Open a database (2 fields), append 50000 records using a loop [Math time]{[time]}-{[start_time]}[/Math]
Replaces 50000 records [Math time]{[time]}-{[start_time]}[/Math]
ReplaceFoundItems 50000 records [Math time]{[time]}-{[start_time]}[/Math]
Just delete the records [Math time]{[time]}-{[start_time]}[/Math]
global time: [Math time]{[time]}-{[start_time_global]}[/Math] ---------------------------------------------------------------------- - chris On Jul 19, 2010, at 9:25, Kenneth Grome wrote: > Here's the code in my trigger template which is requested once per = second. Instead of trying to create 45 new records in a couple of = seconds I have resorted to creating only 1-4 new records per second = (although I want to create many more for a truly realistic auction = simulation). >=20 > The tickets.db holds all the bidderID (sku) values. I need to search = this db and retrieve a series of random bidderID's for the replace = contexts: >=20 >=20 > *************************************************************** >=20 > > [hideif [getchars start=3D4&end=3D5][time][/getchars]\2] > [showif [getchars start=3D7&end=3D8][time][/getchars]\11] > [search = db=3Dtest/tickets.db&eqemaildatarq=3DXXX&raemailsort=3D1&max=3D[math]ceil(= (1+[random])/25)[/math]][founditems] > [replace = db=3Dtest/bids.db&eqbidderIDdatarq=3D[blank]&asidxsort=3D1&idxtype=3Dnum&m= ax=3D1]bidTime=3D[time]&bidderID=3D[sku][/replace] > [/founditems][/search] > [/showif] > [/hideif] >=20 > > [showif [getchars start=3D4&end=3D5][time][/getchars]\6] > [showif [getchars start=3D7&end=3D8][time][/getchars]\17] > [search = db=3Dtest/tickets.db&eqemaildatarq=3DXXX&raemailsort=3D1&max=3D[math]ceil(= (1+[random])/25)[/math]][founditems] > [replace = db=3Dtest/bids.db&eqbidderIDdatarq=3D[blank]&asidxsort=3D1&idxtype=3Dnum&m= ax=3D1]bidTime=3D[time]&bidderID=3D[sku][/replace] > [/founditems][/search] > [/showif] > [/showif] >=20 > *************************************************************** >=20 >=20 > Naturally I cannot use this technique with replacefounditems because = I'm searching in a different db than where the records need to be = replaced. But there may be another way to approach this problem. I = think this might work faster: >=20 > 1- Search the tickets.db for (example) 45 random sku values > 2. Stores these skus in an indexed table > 3- Search the bids.db for the next 45 records with blank bidderID = values > 4- Use replacefounditems to change these bidderID's based on lookups = of the index values in the table >=20 > I don't know if this will be faster or not, but it's probably worth a = try as soon as I have some free time again. >=20 > Sincerely, > Kenneth Grome >=20 >=20 >=20 > --------------------------------------------------------- > This message is sent to you because you are subscribed to > the mailing list . > To unsubscribe, E-mail to: > archives: http://mail.webdna.us/list/talk@webdna.us > Bug Reporting: support@webdna.us Associated Messages, from the most recent to the oldest:

    
  1. Re: [WebDNA] CORRECTION: 60% failure rate using replace in a loop (christophe.billiottet@webdna.us 2010)
  2. Re: [WebDNA] CORRECTION: 60% failure rate using replace in a loop (Kenneth Grome 2010)
  3. Re: [WebDNA] CORRECTION: 60% failure rate using replace in a loop (Kenneth Grome 2010)
  4. Re: [WebDNA] CORRECTION: 60% failure rate using replace in a loop (christophe.billiottet@webdna.us 2010)
  5. Re: [WebDNA] CORRECTION: 60% failure rate using replace in a loop (Kenneth Grome 2010)
  6. Re: [WebDNA] CORRECTION: 60% failure rate using replace in a loop (Kenneth Grome 2010)
  7. Re: [WebDNA] CORRECTION: 60% failure rate using replace in a loop (christophe.billiottet@webdna.us 2010)
  8. Re: [WebDNA] CORRECTION: 60% failure rate using replace in a loop (Kenneth Grome 2010)
  9. Re: [WebDNA] CORRECTION: 60% failure rate using replace in a loop (christophe.billiottet@webdna.us 2010)
  10. Re: [WebDNA] CORRECTION: 60% failure rate using replace in a loop (Kenneth Grome 2010)
  11. Re: [WebDNA] CORRECTION: 60% failure rate using replace in a loop (christophe.billiottet@webdna.us 2010)
  12. Re: [WebDNA] CORRECTION: 60% failure rate using replace in a loop (Kenneth Grome 2010)
  13. Re: [WebDNA] CORRECTION: 60% failure rate using replace in a loop (christophe.billiottet@webdna.us 2010)
  14. [WebDNA] CORRECTION: 60% failure rate using replace in a loop (Kenneth Grome 2010)
Ken, something else: do you use your code locally? on a remote server? = using WebDNA.fcgi 7.0? with accelerator activated? i am doing some = basic tests on an accelerated WebDNA.fcgi 7.0 (local) and so far, i can = append 4,500 records per second and replacing 50,000 records takes less = than a second. Here my results Open a database (2 fields), append 50000 records using a loop 00:00:13=20= Replaces 50000 records 00:00:00=20 ReplaceFoundItems 50000 records 00:00:00=20 Just delete the records 00:00:00=20 global time: 00:00:13 Here the test code: ---------------------------------------------------------------------- [!] ##Check for the Database first - create it if not there--[/!] [showif T=3D[fileinfo file=3Dspeed.txt][exists][/fileinfo]] [closedatabase db=3Dspeed.txt] [deletefile file=3Dspeed.txt] [/showif] [showif F=3D[fileinfo file=3Dspeed.txt][exists][/fileinfo]] [writefile file=3Dspeed.txt]skuvalue [/writefile] [/showif] [!] ##Delete the records in the DB to ensure the search is 'clean'= --[/!] [delete db=3Dspeed.txt&neSKUdata=3Dfind_all] [text]start_time_global=3D[time][/text]
Open a database (2 fields), append 50000 records using a loop [Math time]{[time]}-{[start_time]}[/Math]
Replaces 50000 records [Math time]{[time]}-{[start_time]}[/Math]
ReplaceFoundItems 50000 records [Math time]{[time]}-{[start_time]}[/Math]
Just delete the records [Math time]{[time]}-{[start_time]}[/Math]
global time: [Math time]{[time]}-{[start_time_global]}[/Math] ---------------------------------------------------------------------- - chris On Jul 19, 2010, at 9:25, Kenneth Grome wrote: > Here's the code in my trigger template which is requested once per = second. Instead of trying to create 45 new records in a couple of = seconds I have resorted to creating only 1-4 new records per second = (although I want to create many more for a truly realistic auction = simulation). >=20 > The tickets.db holds all the bidderID (sku) values. I need to search = this db and retrieve a series of random bidderID's for the replace = contexts: >=20 >=20 > *************************************************************** >=20 > > [hideif [getchars start=3D4&end=3D5][time][/getchars]\2] > [showif [getchars start=3D7&end=3D8][time][/getchars]\11] > [search = db=3Dtest/tickets.db&eqemaildatarq=3DXXX&raemailsort=3D1&max=3D[math]ceil(= (1+[random])/25)[/math]][founditems] > [replace = db=3Dtest/bids.db&eqbidderIDdatarq=3D[blank]&asidxsort=3D1&idxtype=3Dnum&m= ax=3D1]bidTime=3D[time]&bidderID=3D[sku][/replace] > [/founditems][/search] > [/showif] > [/hideif] >=20 > > [showif [getchars start=3D4&end=3D5][time][/getchars]\6] > [showif [getchars start=3D7&end=3D8][time][/getchars]\17] > [search = db=3Dtest/tickets.db&eqemaildatarq=3DXXX&raemailsort=3D1&max=3D[math]ceil(= (1+[random])/25)[/math]][founditems] > [replace = db=3Dtest/bids.db&eqbidderIDdatarq=3D[blank]&asidxsort=3D1&idxtype=3Dnum&m= ax=3D1]bidTime=3D[time]&bidderID=3D[sku][/replace] > [/founditems][/search] > [/showif] > [/showif] >=20 > *************************************************************** >=20 >=20 > Naturally I cannot use this technique with replacefounditems because = I'm searching in a different db than where the records need to be = replaced. But there may be another way to approach this problem. I = think this might work faster: >=20 > 1- Search the tickets.db for (example) 45 random sku values > 2. Stores these skus in an indexed table > 3- Search the bids.db for the next 45 records with blank bidderID = values > 4- Use replacefounditems to change these bidderID's based on lookups = of the index values in the table >=20 > I don't know if this will be faster or not, but it's probably worth a = try as soon as I have some free time again. >=20 > Sincerely, > Kenneth Grome >=20 >=20 >=20 > --------------------------------------------------------- > This message is sent to you because you are subscribed to > the mailing list . > To unsubscribe, E-mail to: > archives: http://mail.webdna.us/list/talk@webdna.us > Bug Reporting: support@webdna.us christophe.billiottet@webdna.us

DOWNLOAD WEBDNA NOW!

Top Articles:

Talk List

The WebDNA community talk-list is the best place to get some help: several hundred extremely proficient programmers with an excellent knowledge of WebDNA and an excellent spirit will deliver all the tips and tricks you can imagine...

Related Readings:

How to archive....? (1998) shipcost (1997) Alternating BGColors in Table Rows (1998) Quit revisited (1997) help with writefile (1998) Any help gratefully appreciated (2001) image attachment (2002) Sorting by date (1997) WebMerchant still not working!! (1999) Printing controls - form feeds (2002) taxTotal, grandTotal (1997) user authentication via TCPSend (2005) Suffix or Line? (1999) Follow-up to listfiles bug report ... (2003) Sku numbers (1997) [Sum] function? (1997) find with exceptions (1997) Email Set-Up? (1997) More to Expire (1998) Special characters & [sendmail] (1998)