Re: [WebDNA] two ideas for running a cluster of WebDNA servers
This WebDNA talk-list message is from 2019
It keeps the original formatting.
numero = 114880
interpreted = N
texte = 2508 I=E2=80=99m very interested in this. At times we have hundreds of devices a minute hitting our server.=20 The only thing that has made performance better is investing in new = hardware that can deal with the load. (Fast SSDs made a huge difference) A singular store for databases would be the only way I can think of to = achieve this across endpoints. - Greg > On Oct 5, 2019, at 10:42 AM, WebDNA Solutions =
wrote: >=20 >> One front-end server that will redirect the requests >> through POST or GET to a cluster of back-end servers. When >> the request is for writing, then only one of the back-end >> servers is solicited (always the same), when is is for >> reading, it could be any of the back-end servers. >>=20 >> The "writing" server is the master, the others are slaves. >> Every 30 sec, through rsync, the master is copied to a >> slave, and the WebDNA slave reloads the databases in RAM. >=20 > Actually in this situation the WebDNA dbs on the slave servers > must be flushed immediately BEFORE each rsync overwrites > them. If this happens every 30 seconds it will effectively > defeat one of the main advantages of WebDNA -- its typically > superfast "read from RAM" performance -- because every updated > db would have to be read into RAM agai every 30 seconds. >=20 >=20 >=20 >> Another idea would be the same front-end server with the >> same backend servers, and instead of using rsync, the front >> end server would POST the request to each one of the >> back-end servers: no more master and slave. >=20 > I did something similar many years ago, but instead of > frontend and backend servers each server was equally > available via a sequential round-robin load distribution > system. This eliminated the need to flush the dbs every time > one server posted an update to another server, because they > would all be posted to at virtually the same time (within a > fraction of a second from each other given a fast enough > network connection). >=20 > But when the internet connection between the servers goes > down it creates SERIOUS out-of-sync problems such as failure > to login, failure to see the updates the visitors just made, > etc. ... and this would absolutely require your next idea in > order to rebuild the dbs correctly when necessary: >=20 >> Another idea would be to keep a log database of all the >> writing requests so, by reading this log file, all the >> "slaves" would get the same information. Databases could >> even be rebuilt in case of necessity. >=20 > A system like this would become very complex and error-prone > in my opinion, thus unlikely worth the effort when non-WebDNA > solutions are readily available and have been tested and > proven in terms of reliability for many years already. >=20 >=20 >> I am sure there are other solutions. Any other idea? >=20 > WebDNA's design is ideal for use on a single frontend server. > To try and turn it into a backend server would probably suck, > not just because of the amount of work required in an attempt > to turn it into something it is not, but mostly because if it > loses the inherent advantages it offers as a RAM-resident > frontend db server you'll end up with a system that > underperforms the competition. >=20 > I'm wondering why you're thinking about this? >=20 > Are you trying to come up with a plan to "add value" to > WebDNA and perhaps broaden its market by attempting to make > WebDNA a suitable system for websites that need more than one > server in order to handle extreme loads? >=20 >=20 > Regards, > Kenneth Grome > WebDNA Solutions > http://www.webdnasolutions.com > Urgent/Emergency Phone: (228) 222-2917 > Website, Database, Network, and Communication Systems >=20 >=20 >=20 > --------------------------------------------------------- > This message is sent to you because you are subscribed to > the mailing list talk@webdna.us > To unsubscribe, E-mail to: talk-leave@webdna.us > archives: http://www.webdna.us/page.dna?numero=3D55 > Bug Reporting: support@webdna.us --------------------------------------------------------- This message is sent to you because you are subscribed to the mailing list talk@webdna.us To unsubscribe, E-mail to: talk-leave@webdna.us archives: http://www.webdna.us/page.dna?numero=3D55 Bug Reporting: support@webdna.us .
Associated Messages, from the most recent to the oldest:
2508 I=E2=80=99m very interested in this. At times we have hundreds of devices a minute hitting our server.=20 The only thing that has made performance better is investing in new = hardware that can deal with the load. (Fast SSDs made a huge difference) A singular store for databases would be the only way I can think of to = achieve this across endpoints. - Greg > On Oct 5, 2019, at 10:42 AM, WebDNA Solutions = wrote: >=20 >> One front-end server that will redirect the requests >> through POST or GET to a cluster of back-end servers. When >> the request is for writing, then only one of the back-end >> servers is solicited (always the same), when is is for >> reading, it could be any of the back-end servers. >>=20 >> The "writing" server is the master, the others are slaves. >> Every 30 sec, through rsync, the master is copied to a >> slave, and the WebDNA slave reloads the databases in RAM. >=20 > Actually in this situation the WebDNA dbs on the slave servers > must be flushed immediately BEFORE each rsync overwrites > them. If this happens every 30 seconds it will effectively > defeat one of the main advantages of WebDNA -- its typically > superfast "read from RAM" performance -- because every updated > db would have to be read into RAM agai every 30 seconds. >=20 >=20 >=20 >> Another idea would be the same front-end server with the >> same backend servers, and instead of using rsync, the front >> end server would POST the request to each one of the >> back-end servers: no more master and slave. >=20 > I did something similar many years ago, but instead of > frontend and backend servers each server was equally > available via a sequential round-robin load distribution > system. This eliminated the need to flush the dbs every time > one server posted an update to another server, because they > would all be posted to at virtually the same time (within a > fraction of a second from each other given a fast enough > network connection). >=20 > But when the internet connection between the servers goes > down it creates SERIOUS out-of-sync problems such as failure > to login, failure to see the updates the visitors just made, > etc. ... and this would absolutely require your next idea in > order to rebuild the dbs correctly when necessary: >=20 >> Another idea would be to keep a log database of all the >> writing requests so, by reading this log file, all the >> "slaves" would get the same information. Databases could >> even be rebuilt in case of necessity. >=20 > A system like this would become very complex and error-prone > in my opinion, thus unlikely worth the effort when non-WebDNA > solutions are readily available and have been tested and > proven in terms of reliability for many years already. >=20 >=20 >> I am sure there are other solutions. Any other idea? >=20 > WebDNA's design is ideal for use on a single frontend server. > To try and turn it into a backend server would probably suck, > not just because of the amount of work required in an attempt > to turn it into something it is not, but mostly because if it > loses the inherent advantages it offers as a RAM-resident > frontend db server you'll end up with a system that > underperforms the competition. >=20 > I'm wondering why you're thinking about this? >=20 > Are you trying to come up with a plan to "add value" to > WebDNA and perhaps broaden its market by attempting to make > WebDNA a suitable system for websites that need more than one > server in order to handle extreme loads? >=20 >=20 > Regards, > Kenneth Grome > WebDNA Solutions > http://www.webdnasolutions.com > Urgent/Emergency Phone: (228) 222-2917 > Website, Database, Network, and Communication Systems >=20 >=20 >=20 > --------------------------------------------------------- > This message is sent to you because you are subscribed to > the mailing list talk@webdna.us > To unsubscribe, E-mail to: talk-leave@webdna.us > archives: http://www.webdna.us/page.dna?numero=3D55 > Bug Reporting: support@webdna.us --------------------------------------------------------- This message is sent to you because you are subscribed to the mailing list talk@webdna.us To unsubscribe, E-mail to: talk-leave@webdna.us archives: http://www.webdna.us/page.dna?numero=3D55 Bug Reporting: support@webdna.us .
Greg Nelson
DOWNLOAD WEBDNA NOW!
Top Articles:
Talk List
The WebDNA community talk-list is the best place to get some help: several hundred extremely proficient programmers with an excellent knowledge of WebDNA and an excellent spirit will deliver all the tips and tricks you can imagine...
Related Readings:
Newbie problem blah blah blah (1997)
PCS Frames (1997)
WebCatalog can't find database (1997)
Search for value or blank? (2004)
Demo math questions (2000)
Time Tracking (2003)
nslookup (2000)
Reminder for beta testers (2000)
Generating Report Totals (1997)
Mac: LModelDirector bug fix (1997)
RE: Languages (1997)
Authenticating users without dialog box (1997)
BIG PROBLEMS with Checkboxes!!! (1998)
[OT] Google Info (2004)
WebCatalog can't find database (1997)
Nested tags count question (1997)
PowerKey and PageSentry not playing nice (2000)
Re[3]: 2nd WebCatalog2 Feature Request (1996)
Multi Colonns (1998)
Change Subtotal (2000)