Re: [WebDNA] two ideas for running a cluster of WebDNA servers
This WebDNA talk-list message is from 2019
It keeps the original formatting.
numero = 114880
interpreted = N
texte = 2508 I=E2=80=99m very interested in this. At times we have hundreds of devices a minute hitting our server.=20 The only thing that has made performance better is investing in new = hardware that can deal with the load. (Fast SSDs made a huge difference) A singular store for databases would be the only way I can think of to = achieve this across endpoints. - Greg > On Oct 5, 2019, at 10:42 AM, WebDNA Solutions =
wrote: >=20 >> One front-end server that will redirect the requests >> through POST or GET to a cluster of back-end servers. When >> the request is for writing, then only one of the back-end >> servers is solicited (always the same), when is is for >> reading, it could be any of the back-end servers. >>=20 >> The "writing" server is the master, the others are slaves. >> Every 30 sec, through rsync, the master is copied to a >> slave, and the WebDNA slave reloads the databases in RAM. >=20 > Actually in this situation the WebDNA dbs on the slave servers > must be flushed immediately BEFORE each rsync overwrites > them. If this happens every 30 seconds it will effectively > defeat one of the main advantages of WebDNA -- its typically > superfast "read from RAM" performance -- because every updated > db would have to be read into RAM agai every 30 seconds. >=20 >=20 >=20 >> Another idea would be the same front-end server with the >> same backend servers, and instead of using rsync, the front >> end server would POST the request to each one of the >> back-end servers: no more master and slave. >=20 > I did something similar many years ago, but instead of > frontend and backend servers each server was equally > available via a sequential round-robin load distribution > system. This eliminated the need to flush the dbs every time > one server posted an update to another server, because they > would all be posted to at virtually the same time (within a > fraction of a second from each other given a fast enough > network connection). >=20 > But when the internet connection between the servers goes > down it creates SERIOUS out-of-sync problems such as failure > to login, failure to see the updates the visitors just made, > etc. ... and this would absolutely require your next idea in > order to rebuild the dbs correctly when necessary: >=20 >> Another idea would be to keep a log database of all the >> writing requests so, by reading this log file, all the >> "slaves" would get the same information. Databases could >> even be rebuilt in case of necessity. >=20 > A system like this would become very complex and error-prone > in my opinion, thus unlikely worth the effort when non-WebDNA > solutions are readily available and have been tested and > proven in terms of reliability for many years already. >=20 >=20 >> I am sure there are other solutions. Any other idea? >=20 > WebDNA's design is ideal for use on a single frontend server. > To try and turn it into a backend server would probably suck, > not just because of the amount of work required in an attempt > to turn it into something it is not, but mostly because if it > loses the inherent advantages it offers as a RAM-resident > frontend db server you'll end up with a system that > underperforms the competition. >=20 > I'm wondering why you're thinking about this? >=20 > Are you trying to come up with a plan to "add value" to > WebDNA and perhaps broaden its market by attempting to make > WebDNA a suitable system for websites that need more than one > server in order to handle extreme loads? >=20 >=20 > Regards, > Kenneth Grome > WebDNA Solutions > http://www.webdnasolutions.com > Urgent/Emergency Phone: (228) 222-2917 > Website, Database, Network, and Communication Systems >=20 >=20 >=20 > --------------------------------------------------------- > This message is sent to you because you are subscribed to > the mailing list talk@webdna.us > To unsubscribe, E-mail to: talk-leave@webdna.us > archives: http://www.webdna.us/page.dna?numero=3D55 > Bug Reporting: support@webdna.us --------------------------------------------------------- This message is sent to you because you are subscribed to the mailing list talk@webdna.us To unsubscribe, E-mail to: talk-leave@webdna.us archives: http://www.webdna.us/page.dna?numero=3D55 Bug Reporting: support@webdna.us .
Associated Messages, from the most recent to the oldest:
2508 I=E2=80=99m very interested in this. At times we have hundreds of devices a minute hitting our server.=20 The only thing that has made performance better is investing in new = hardware that can deal with the load. (Fast SSDs made a huge difference) A singular store for databases would be the only way I can think of to = achieve this across endpoints. - Greg > On Oct 5, 2019, at 10:42 AM, WebDNA Solutions = wrote: >=20 >> One front-end server that will redirect the requests >> through POST or GET to a cluster of back-end servers. When >> the request is for writing, then only one of the back-end >> servers is solicited (always the same), when is is for >> reading, it could be any of the back-end servers. >>=20 >> The "writing" server is the master, the others are slaves. >> Every 30 sec, through rsync, the master is copied to a >> slave, and the WebDNA slave reloads the databases in RAM. >=20 > Actually in this situation the WebDNA dbs on the slave servers > must be flushed immediately BEFORE each rsync overwrites > them. If this happens every 30 seconds it will effectively > defeat one of the main advantages of WebDNA -- its typically > superfast "read from RAM" performance -- because every updated > db would have to be read into RAM agai every 30 seconds. >=20 >=20 >=20 >> Another idea would be the same front-end server with the >> same backend servers, and instead of using rsync, the front >> end server would POST the request to each one of the >> back-end servers: no more master and slave. >=20 > I did something similar many years ago, but instead of > frontend and backend servers each server was equally > available via a sequential round-robin load distribution > system. This eliminated the need to flush the dbs every time > one server posted an update to another server, because they > would all be posted to at virtually the same time (within a > fraction of a second from each other given a fast enough > network connection). >=20 > But when the internet connection between the servers goes > down it creates SERIOUS out-of-sync problems such as failure > to login, failure to see the updates the visitors just made, > etc. ... and this would absolutely require your next idea in > order to rebuild the dbs correctly when necessary: >=20 >> Another idea would be to keep a log database of all the >> writing requests so, by reading this log file, all the >> "slaves" would get the same information. Databases could >> even be rebuilt in case of necessity. >=20 > A system like this would become very complex and error-prone > in my opinion, thus unlikely worth the effort when non-WebDNA > solutions are readily available and have been tested and > proven in terms of reliability for many years already. >=20 >=20 >> I am sure there are other solutions. Any other idea? >=20 > WebDNA's design is ideal for use on a single frontend server. > To try and turn it into a backend server would probably suck, > not just because of the amount of work required in an attempt > to turn it into something it is not, but mostly because if it > loses the inherent advantages it offers as a RAM-resident > frontend db server you'll end up with a system that > underperforms the competition. >=20 > I'm wondering why you're thinking about this? >=20 > Are you trying to come up with a plan to "add value" to > WebDNA and perhaps broaden its market by attempting to make > WebDNA a suitable system for websites that need more than one > server in order to handle extreme loads? >=20 >=20 > Regards, > Kenneth Grome > WebDNA Solutions > http://www.webdnasolutions.com > Urgent/Emergency Phone: (228) 222-2917 > Website, Database, Network, and Communication Systems >=20 >=20 >=20 > --------------------------------------------------------- > This message is sent to you because you are subscribed to > the mailing list talk@webdna.us > To unsubscribe, E-mail to: talk-leave@webdna.us > archives: http://www.webdna.us/page.dna?numero=3D55 > Bug Reporting: support@webdna.us --------------------------------------------------------- This message is sent to you because you are subscribed to the mailing list talk@webdna.us To unsubscribe, E-mail to: talk-leave@webdna.us archives: http://www.webdna.us/page.dna?numero=3D55 Bug Reporting: support@webdna.us .
Greg Nelson
DOWNLOAD WEBDNA NOW!
Top Articles:
Talk List
The WebDNA community talk-list is the best place to get some help: several hundred extremely proficient programmers with an excellent knowledge of WebDNA and an excellent spirit will deliver all the tips and tricks you can imagine...
Related Readings:
Questions To Answer (1997)
[WebDNA] Rewrite url (2013)
WebMerchant & CC Response (2002)
Just Testing (1997)
Some Questions (1997)
A Global Variable (1997)
Search returns all, not 20 (1997)
Default admin bug and how tech support reacted to it. (2000)
Discounts (2002)
[WebDNA] remove html comments (2009)
[isfolder] and [filename] (1997)
[cart] (1998)
search question (2001)
tcpconnect return (2003)
PC Cookie Problem? (2003)
autosensing lanague selection (1997)
[addlineitems] display (1997)
Getting Emailer to send mail (1997)
problem serving foreign languages text (1997)
UPDATE PROBLEM (1997)