Re: Here we go again...WebDNA - SQL- Clustering
This WebDNA talk-list message is from 2006
It keeps the original formatting.
numero = 66741
interpreted = N
texte = Oh boy... I like the last joke! -----Original Message----- From: WebDNA Talk [mailto:WebDNA-Talk@talk.smithmicro.com]On Behalf Of Alex McCombie Sent: Tuesday, April 04, 2006 12:00 PM To: WebDNA Talk Subject: Re: Here we go again...WebDNA - SQL- Clustering >> Again flat file has limitation on performance on server >=20 > Bess, what you fail to mention is that a flat file database structure > can function just perfectly (with no limitations) for the majority > of needs out there. Sure, it depends on what you are targeting. WebDNA > does eCommerce, but it does LAN stuff and other functions as well. = These > blanket statements are just wrong. I don't know about majority because that too is debatable, but suffice = to say that Donovan is right and that is works very well in numerous = projects. Like i have said in the past postings, we use them interchangeably = though I must admit that I move to SQL tables instinctively now. The other thing missing here is that WebDNA is first and foremost a markup language and = not a database system! And as a MUL it just plain rocks. My developers come = to the table with php and .net experience and they LOVE, repeat LOVE DNA = for the simplicity and small footprint the code makes. Very clean and quick = to get to the goal... MySQL, MSSQL or DNA flat files alike. =20 >> Flat file makes it too much hassle to do any business intelligences, > data mining or even reporting. It is not scalable for enterprise = application. > Just flat out wrong. You are newbie to this list and to say webdna = can't > do reporting etc.. is just silly. Absolutely correct. Data mining and reporting are all easily done in DNA tables. She's right about the scalability issue though. I cant even tell = you how many times the conversation about scaling multiple DNA installations across multiple servers has been hashed and rehashed on this list over = the years by the best on the list. The truth is that is always fell short - = way short -- for us until we started fronting SQL with DNA. Now scalable is = just a given and WebDNA has been extended for us where we were facing the decision to have to move away. =20 >> Thanks Kenneth. Thanks for clarifying the myth. > Kenneth is a Myth... O.K., I just had to put that one in there for > enternainment > purposes. Now that's funny. I think even Ken would agree! :-) =20 >> I think it is important for developers to understand the facts. = WebDNA is not >> "relational" database > Again, that's just plain wrong. It doesn't take a skilled developer to = do > this. > [search db=3D...&blah] > [founditems][lookup > = db=3Danother&value=3DID_OF_SECOND_DB_STORED_IN_FIRST_DATABASE&Blah]etc...= [/foundit > ems] > [/search] Its true that we build the relationships like Donovan mentioned above = but I can tell you that I have found better performance (especially on large = data sets) with the join capability native to SQL than recursive searching = could do. On small datasets the recursive searching is ok but I have seen = outputs where in order to show multiple columns of data 50 rows deep (which is = not unreasonable) with 5-6 columns of related data -- where you want to show = the titles and not the related sku numbers... That led to recursive searches that number 300+ per page. Did it do it? Sure. Did it do it adequately on small data sets? You bet. = But once you try that on lets say a discussion forum, with 200,000 records = and tons of text, that recursive searches really starts to show its = limitations. This is just one real world (and not unreasonable) example where join searching on SQL is much faster. In fact I had to go back and = de-normalize my data to put calculated totals into fields so as not to have to do the numfound searching in a recursive lookup. It worked but it wasn't = elegant by any means. One other small point, though in the bigger context it is a smaller = point, by fronting DNA out to something like SQL you move the performance = sucking work (searching) to another box or boxes dedicated to DB serving. Unfortunately with the built in RAM resident only access of DNA flat = files, it all has to happen on the same box as the webserver, which may or may = not have it's own performance issues. Scalability is the huge winner in this case. =20 >> Tab-delimited text files... It really makes many non-webdna = developers >> "wonder" > You make it sound like a text table is not ever used on the web. Think = Again! I got the same look all the time. Few of my clients came without their = own IT dept. though. That being said, few wouldn't have some sort of = exposure to things like SQL, SAP, etc. Flat files was something that we learned to = avoid in the conversations even though we kknew it would work for them. Now we simply say everything is run from MSSQL and we can hit it with a variety = of interfaces. Our interface of choice is WebDNA because we can accomplish = the same tasks in typically 1/2 the time and save you money.... Yet your = data is right there for any other future development/growth/reporting/etc. :-) Everyone wins. (including SMSI) >> Also once your flat file grow beyond roughly 2 GB (check with Network = Admin >> to verify the size), > This is a limit set by the OS and hard drive formatting... not webdna. >> you can't run data very well. The go-around solution is to break down = WebDNA >> database into smaller >> chunks for faster performance. >=20 > Its not a "go-around" its just good programming, and in fact, is = called a > relational framework. Yes and no. Again, even with full normalization and excellent data structuring skills you will still see data sets that large in the most innocent of places. I like to pick on discussion forums because I run = many. And at a couple hundred thousand records its starts to show it girth and = has stability issues. Point is you cannot always employ workarounds, or = good database design to make your data 'smaller'. I have a hobby site that = gets 3 million connections per month and 90% of that is discussion forum = activity. It has survived in DNA flat tables for years but is currently being redeveloped in DNA/MSSQL. I expect to see great returns in performance, stability, expandability. BUT MY FRONT END IS STILL WEBDNA MARKUP = LANGUAGE! Because that's ultimately what WebDNA is a kick ass markup language. =20 >> You really need Relationship Database for scaleable application. > Sigh. No you don't. You *may* need it.. but not always. It depends on = what the > client is doing. Definitely true. Just one man's opinion... Not trying to force it on anyone :-) Hope it helps to inform some and doesn't annoy others ;-) --=20 Alex J McCombie =20 Adventure Skies Interactive (ASI) Alex@Adventureskies.com ------------------------------------------------------------- This message is sent to you because you are subscribed to the mailing list
. To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to = Web Archive of this list is at: http://webdna.smithmicro.com/ ------------------------------------------------------------- This message is sent to you because you are subscribed to the mailing list . To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to Web Archive of this list is at: http://webdna.smithmicro.com/
Associated Messages, from the most recent to the oldest:
Oh boy... I like the last joke! -----Original Message----- From: WebDNA Talk [mailto:WebDNA-Talk@talk.smithmicro.com]On Behalf Of Alex McCombie Sent: Tuesday, April 04, 2006 12:00 PM To: WebDNA Talk Subject: Re: Here we go again...WebDNA - SQL- Clustering >> Again flat file has limitation on performance on server >=20 > Bess, what you fail to mention is that a flat file database structure > can function just perfectly (with no limitations) for the majority > of needs out there. Sure, it depends on what you are targeting. WebDNA > does eCommerce, but it does LAN stuff and other functions as well. = These > blanket statements are just wrong. I don't know about majority because that too is debatable, but suffice = to say that Donovan is right and that is works very well in numerous = projects. Like i have said in the past postings, we use them interchangeably = though I must admit that I move to SQL tables instinctively now. The other thing missing here is that WebDNA is first and foremost a markup language and = not a database system! And as a MUL it just plain rocks. My developers come = to the table with php and .net experience and they LOVE, repeat LOVE DNA = for the simplicity and small footprint the code makes. Very clean and quick = to get to the goal... MySQL, MSSQL or DNA flat files alike. =20 >> Flat file makes it too much hassle to do any business intelligences, > data mining or even reporting. It is not scalable for enterprise = application. > Just flat out wrong. You are newbie to this list and to say webdna = can't > do reporting etc.. is just silly. Absolutely correct. Data mining and reporting are all easily done in DNA tables. She's right about the scalability issue though. I cant even tell = you how many times the conversation about scaling multiple DNA installations across multiple servers has been hashed and rehashed on this list over = the years by the best on the list. The truth is that is always fell short - = way short -- for us until we started fronting SQL with DNA. Now scalable is = just a given and WebDNA has been extended for us where we were facing the decision to have to move away. =20 >> Thanks Kenneth. Thanks for clarifying the myth. > Kenneth is a Myth... O.K., I just had to put that one in there for > enternainment > purposes. Now that's funny. I think even Ken would agree! :-) =20 >> I think it is important for developers to understand the facts. = WebDNA is not >> "relational" database > Again, that's just plain wrong. It doesn't take a skilled developer to = do > this. > [search db=3D...&blah] > [founditems][lookup > = db=3Danother&value=3DID_OF_SECOND_DB_STORED_IN_FIRST_DATABASE&Blah]etc...= [/foundit > ems] > [/search] Its true that we build the relationships like Donovan mentioned above = but I can tell you that I have found better performance (especially on large = data sets) with the join capability native to SQL than recursive searching = could do. On small datasets the recursive searching is ok but I have seen = outputs where in order to show multiple columns of data 50 rows deep (which is = not unreasonable) with 5-6 columns of related data -- where you want to show = the titles and not the related sku numbers... That led to recursive searches that number 300+ per page. Did it do it? Sure. Did it do it adequately on small data sets? You bet. = But once you try that on lets say a discussion forum, with 200,000 records = and tons of text, that recursive searches really starts to show its = limitations. This is just one real world (and not unreasonable) example where join searching on SQL is much faster. In fact I had to go back and = de-normalize my data to put calculated totals into fields so as not to have to do the numfound searching in a recursive lookup. It worked but it wasn't = elegant by any means. One other small point, though in the bigger context it is a smaller = point, by fronting DNA out to something like SQL you move the performance = sucking work (searching) to another box or boxes dedicated to DB serving. Unfortunately with the built in RAM resident only access of DNA flat = files, it all has to happen on the same box as the webserver, which may or may = not have it's own performance issues. Scalability is the huge winner in this case. =20 >> Tab-delimited text files... It really makes many non-webdna = developers >> "wonder" > You make it sound like a text table is not ever used on the web. Think = Again! I got the same look all the time. Few of my clients came without their = own IT dept. though. That being said, few wouldn't have some sort of = exposure to things like SQL, SAP, etc. Flat files was something that we learned to = avoid in the conversations even though we kknew it would work for them. Now we simply say everything is run from MSSQL and we can hit it with a variety = of interfaces. Our interface of choice is WebDNA because we can accomplish = the same tasks in typically 1/2 the time and save you money.... Yet your = data is right there for any other future development/growth/reporting/etc. :-) Everyone wins. (including SMSI) >> Also once your flat file grow beyond roughly 2 GB (check with Network = Admin >> to verify the size), > This is a limit set by the OS and hard drive formatting... not webdna. >> you can't run data very well. The go-around solution is to break down = WebDNA >> database into smaller >> chunks for faster performance. >=20 > Its not a "go-around" its just good programming, and in fact, is = called a > relational framework. Yes and no. Again, even with full normalization and excellent data structuring skills you will still see data sets that large in the most innocent of places. I like to pick on discussion forums because I run = many. And at a couple hundred thousand records its starts to show it girth and = has stability issues. Point is you cannot always employ workarounds, or = good database design to make your data 'smaller'. I have a hobby site that = gets 3 million connections per month and 90% of that is discussion forum = activity. It has survived in DNA flat tables for years but is currently being redeveloped in DNA/MSSQL. I expect to see great returns in performance, stability, expandability. BUT MY FRONT END IS STILL WEBDNA MARKUP = LANGUAGE! Because that's ultimately what WebDNA is a kick ass markup language. =20 >> You really need Relationship Database for scaleable application. > Sigh. No you don't. You *may* need it.. but not always. It depends on = what the > client is doing. Definitely true. Just one man's opinion... Not trying to force it on anyone :-) Hope it helps to inform some and doesn't annoy others ;-) --=20 Alex J McCombie =20 Adventure Skies Interactive (ASI) Alex@Adventureskies.com ------------------------------------------------------------- This message is sent to you because you are subscribed to the mailing list . To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to = Web Archive of this list is at: http://webdna.smithmicro.com/ ------------------------------------------------------------- This message is sent to you because you are subscribed to the mailing list . To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to Web Archive of this list is at: http://webdna.smithmicro.com/
"Bess Ho"
DOWNLOAD WEBDNA NOW!
Top Articles:
Talk List
The WebDNA community talk-list is the best place to get some help: several hundred extremely proficient programmers with an excellent knowledge of WebDNA and an excellent spirit will deliver all the tips and tricks you can imagine...
Related Readings:
Summ (1998)
Setting variables. (1998)
[isfile] ? (1997)
looking for developers with solutions (2000)
WebCat b13 CGI -shownext- (1997)
WebCatalog for Postcards ? (1997)
RE: Questions To Answer (1997)
Perl programmer needed (2002)
New Command prefs ... (1997)
[WebDNA] How do we tell what's misconfigured? Or which WebDNA 7 version to use? (2011)
why am I getting an authenticate dialog with no [protect]? (2000)
WebCat2b13MacPlugIn - [showif][search][/showif] (1997)
Date AND time (2001)
OK, here goes... (1997)
WC1.6 to WC2 date formatting (1997)
verify email address (1998)
Stopping bad HTML propagation ? (1997)
[WebDNA] Conversion of 500.000 records; advise needed (2008)
Single Link browsing (1997)
Linux file configuration (1999)