Re: [lookup] speed sales pitch
This WebDNA talk-list message is from 2003
It keeps the original formatting.
numero = 53515
interpreted = N
texte = A lookup is essentially the same as a search with "eqFIELDdata=xxx&allcase=T&rank=off&max=1". The lookup is still slightly faster because of less parsing overhead and (possibly) because it can stop as soon as it finds a match without needing to look at the rest of the records (if it doesn't do this, it should). There are more factors involved than just the number of records. How much data are you storing per record? Just the user name (say 16 chars, average) and password (10 chars for US phone number), or are you including contact info, site usage stats, client access options, etc. with each record? There is a sliding point somewhere out there where switching to an SQL db becomes faster than using a WebDNA .db file, depending on the number of records, the size of the records, the complexity of the searches, etc. WebDNA definitely bogs down when the dbs get huge, even if you are loaded with RAM and fast processors, because WebDNA doesn't utilize pre-build indexes and hashing algorithms for direct access to records - basically it needs to look at every record to find a match, so processing more, larger records takes time. With small to medium size databases, WebDNA's RAM-based searches and lookups are fast, but with megalithic databases, SQL is faster. Thus, no definitive answer can be given to whether "this might bog down the server" without more information or some test runs. Were it my project, I would have WebDNA create "4 million" dummy records with sequentially assigned "phone numbers" and reasonable facsimiles of the associated data, then run some performance tests to see how well the lookups perform. - brian On Tuesday, October 21, 2003, at 08:50 AM, Dan Strong wrote: > A client has asked that I build his login scheme so that his clients' > phone numbers are the password ... > He was worried that this might bog down the server since he has '4 > million' phone numbers (hey, that's what he said). > > I told him that the technique [lookup] I am using could handle this > just fine as it is not, technically, 'searching' 4 million phone > numbers at once, but rather 'looking for 1 phone number'. > > I know my explanation was simplistic, but any more detail would have > been lost on him.. > > Is my explanation basically true? Would 4 million (theoretically... > I'm sure he was exagerating...) records bog down a [lookup]? > -- Brian Fries, BrainScan Software -- http://www.brainscansoftware.com -- ------------------------------------------------------------- This message is sent to you because you are subscribed to the mailing list
. To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to Web Archive of this list is at: http://webdna.smithmicro.com/
Associated Messages, from the most recent to the oldest:
A lookup is essentially the same as a search with "eqFIELDdata=xxx&allcase=T&rank=off&max=1". The lookup is still slightly faster because of less parsing overhead and (possibly) because it can stop as soon as it finds a match without needing to look at the rest of the records (if it doesn't do this, it should). There are more factors involved than just the number of records. How much data are you storing per record? Just the user name (say 16 chars, average) and password (10 chars for US phone number), or are you including contact info, site usage stats, client access options, etc. with each record? There is a sliding point somewhere out there where switching to an SQL db becomes faster than using a WebDNA .db file, depending on the number of records, the size of the records, the complexity of the searches, etc. WebDNA definitely bogs down when the dbs get huge, even if you are loaded with RAM and fast processors, because WebDNA doesn't utilize pre-build indexes and hashing algorithms for direct access to records - basically it needs to look at every record to find a match, so processing more, larger records takes time. With small to medium size databases, WebDNA's RAM-based searches and lookups are fast, but with megalithic databases, SQL is faster. Thus, no definitive answer can be given to whether "this might bog down the server" without more information or some test runs. Were it my project, I would have WebDNA create "4 million" dummy records with sequentially assigned "phone numbers" and reasonable facsimiles of the associated data, then run some performance tests to see how well the lookups perform. - brian On Tuesday, October 21, 2003, at 08:50 AM, Dan Strong wrote: > A client has asked that I build his login scheme so that his clients' > phone numbers are the password ... > He was worried that this might bog down the server since he has '4 > million' phone numbers (hey, that's what he said). > > I told him that the technique [lookup] I am using could handle this > just fine as it is not, technically, 'searching' 4 million phone > numbers at once, but rather 'looking for 1 phone number'. > > I know my explanation was simplistic, but any more detail would have > been lost on him.. > > Is my explanation basically true? Would 4 million (theoretically... > I'm sure he was exagerating...) records bog down a [lookup]? > -- Brian Fries, BrainScan Software -- http://www.brainscansoftware.com -- ------------------------------------------------------------- This message is sent to you because you are subscribed to the mailing list . To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to Web Archive of this list is at: http://webdna.smithmicro.com/
Brian Fries
DOWNLOAD WEBDNA NOW!
Top Articles:
Talk List
The WebDNA community talk-list is the best place to get some help: several hundred extremely proficient programmers with an excellent knowledge of WebDNA and an excellent spirit will deliver all the tips and tricks you can imagine...
Related Readings:
Can you specify encoding=multiline? (2000)
minimalist shopping cart. (1997)
Country & Ship-to address & other fields ? (1997)
RE: Deafness (1998)
admin:admin won't work for login:password ! (1999)
POS (2000)
can WC render sites out? (1997)
Re[2]: 2nd WebCatalog2 Feature Request (1996)
RE:DatabaseHelper (1997)
TRAINING videos - Prove IT. (1998)
PCS Frames (1997)
Applescript, WebDNA, and PDF (1997)
Date Calulation (1997)
Strange errors in error log (1998)
Emailer Set Up (1997)
Tax Moratorium Expiring (2001)
WebCat and image maps (1997)
WebCatalog for Postcards ? (1997)
Date math format (1998)
price carry over (1997)