SMSI: databases corrupted on crash and permission issues on OSX

This WebDNA talk-list message is from

2002

It keeps the original formatting. numero = 43619
interpreted = N
texte = We just upgraded the back end of our store (www.tfaw.com) from a G4-867/OS9/Webstar/Webcat3.07 to Dual-1Ghz-DDR/OSX10.2/apache/Webcat4.5. Most things are working well, but we're having a database problem and lost of problems with file permissions.Our sales database has been truncated on two separate occasions today. Problems with this 40+ MB database coincide exactly with WebDNAMonitor entries in system.log noting that webcat has exited and been restarted:Sep 19 10:12:32 tfaw WebDNAMonitor[574]: WebDNA receives signal SIGBUSValSep 19 10:12:32 tfaw WebDNAMonitor[574]: WebDNA with pid [4338] is aliveSep 19 10:12:33 tfaw WebDNAMonitor[574]: WebDNA with pid [4338] is deadSep 19 10:12:33 tfaw WebDNAMonitor[574]: WebDNA restarts with pid [25411]Sep 19 15:31:02 tfaw WebDNAMonitor[574]: WebDNA receives signal SIGTERMalSep 19 15:31:02 tfaw WebDNAMonitor[574]: WebDNA with pid [25411] is aliveSep 19 15:31:03 tfaw WebDNAMonitor[574]: WebDNA with pid [25411] is deadSep 19 15:31:03 tfaw WebDNAMonitor[574]: WebDNA restarts with pid [10584]Sep 19 15:31:04 tfaw WebDNAMonitor[574]: Thread receives terminate signalSep 19 15:31:04 tfaw WebDNAMonitor[574]: WebDNA with pid [10584] is deadSep 19 15:31:04 tfaw WebDNAMonitor[574]: WebDNA restarts with pid [10585]I'm still running my webcatwatch.sh every minute from root's crontab, because I have found some situations where WebDNAMonitor is not able to recover gracefully from a crash (~1 time in 10). One of our other OSX servers still crashes 4-6 times per day, but I'm hoping to partially fix that with a hardware upgrade.We're now watching (debug file enabled - could the filename be any more cumbersome??) to see if a specific template is causing the crash in 4.x when it ran fine for 2 years in 3.x. We're also going to archive some data from this db to speed access and saving of the file.In the meantime I have to ask: why doesn't webcat write databases to a temp file (i.e. .orders.db.tmp) before replacing the original file? If coded robustly, this change should virtually eliminate the possibility of truncating db files.I realize this situation has come up multiple times in the past on this list, and manual backups of crucial databases hourly or more was the only real solution. This is fine except for the size and number of databases we use and the frequency of changes (lots of changes). Yes, I could write a shell script that saves incremental copies of each database and notifies me if the database is suddenly truncated. However, I believe that maintaining database integrity should be the responsibility of the application to the fullest extent possible, and not left as an exercise for each developer to implement their own safety net. Life would be much simpler for everyone if Webcat didn't mangle our databases in the first place, or even better, if it didn't crash in the middle of saving a db...We have heard from several developers on this topic in the past. I would really appreciate a response from SMSI this time around.Permissions: Whenever webcat moves an orderfile, it changes the ownership of that file from the shared developer owner with mode 664 to owner www and the 644 permissions (no group write). User-created shoppingcarts are www/www and mode 664, which is fine until webcat authorizes them, after which they are www/www and mode 644. Our developers often need to manipulate these orderfiles manually for various reasons. The developers are in the www group, and the parent directories (Orders, Completed, etc) all have 770 permission.A couple solutions come to mind:- set up a crontab to chmod all files in these directories every minute (did this already, but it's a sledgehammer solution)- swap out all the [movefile] contexts with [shell]mv oldpath newpath[/shell] contexts to preserve the file settings- make developers login as www (same user as webcat process), but I'm not too hot about the idea of giving user www a valid shell for security reasons.- report this as a potential bug to SMSI and hope they fix it in a future releaseOn a semi-related note: OSX Jaguar client -> Jaguar client file permissions are an equal pain in the a$$. The default permission set through personal file sharing are 600, which of course prevents apache from serving the files unless you force all developers to login as www (which, again, is not desirable). I haven't tried 10.2 server yet, but that's an expensive solution considering the fraction of 10 Server features we would be using. I'm going to try NFS instead of AFP since all our developers are running OSX.Any thoughts on the above?-Dale-------------------------------------------------------------This message is sent to you because you are subscribed to the mailing list .To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to Web Archive of this list is at: http://search.smithmicro.com/ Associated Messages, from the most recent to the oldest:

SMSI: databases corrupted on crash and permission issues on OSX (Dale LaFountain 2002)

We just upgraded the back end of our store (www.tfaw.com) from a G4-867/OS9/Webstar/Webcat3.07 to Dual-1Ghz-DDR/OSX10.2/apache/Webcat4.5. Most things are working well, but we're having a database problem and lost of problems with file permissions.Our sales database has been truncated on two separate occasions today. Problems with this 40+ MB database coincide exactly with WebDNAMonitor entries in system.log noting that webcat has exited and been restarted:Sep 19 10:12:32 tfaw WebDNAMonitor[574]: WebDNA receives signal SIGBUSValSep 19 10:12:32 tfaw WebDNAMonitor[574]: WebDNA with pid [4338] is aliveSep 19 10:12:33 tfaw WebDNAMonitor[574]: WebDNA with pid [4338] is deadSep 19 10:12:33 tfaw WebDNAMonitor[574]: WebDNA restarts with pid [25411]Sep 19 15:31:02 tfaw WebDNAMonitor[574]: WebDNA receives signal SIGTERMalSep 19 15:31:02 tfaw WebDNAMonitor[574]: WebDNA with pid [25411] is aliveSep 19 15:31:03 tfaw WebDNAMonitor[574]: WebDNA with pid [25411] is deadSep 19 15:31:03 tfaw WebDNAMonitor[574]: WebDNA restarts with pid [10584]Sep 19 15:31:04 tfaw WebDNAMonitor[574]: Thread receives terminate signalSep 19 15:31:04 tfaw WebDNAMonitor[574]: WebDNA with pid [10584] is deadSep 19 15:31:04 tfaw WebDNAMonitor[574]: WebDNA restarts with pid [10585]I'm still running my webcatwatch.sh every minute from root's crontab, because I have found some situations where WebDNAMonitor is not able to recover gracefully from a crash (~1 time in 10). One of our other OSX servers still crashes 4-6 times per day, but I'm hoping to partially fix that with a hardware upgrade.We're now watching (debug file enabled - could the filename be any more cumbersome??) to see if a specific template is causing the crash in 4.x when it ran fine for 2 years in 3.x. We're also going to archive some data from this db to speed access and saving of the file.In the meantime I have to ask: why doesn't webcat write databases to a temp file (i.e. .orders.db.tmp) before replacing the original file? If coded robustly, this change should virtually eliminate the possibility of truncating db files.I realize this situation has come up multiple times in the past on this list, and manual backups of crucial databases hourly or more was the only real solution. This is fine except for the size and number of databases we use and the frequency of changes (lots of changes). Yes, I could write a shell script that saves incremental copies of each database and notifies me if the database is suddenly truncated. However, I believe that maintaining database integrity should be the responsibility of the application to the fullest extent possible, and not left as an exercise for each developer to implement their own safety net. Life would be much simpler for everyone if Webcat didn't mangle our databases in the first place, or even better, if it didn't crash in the middle of saving a db...We have heard from several developers on this topic in the past. I would really appreciate a response from SMSI this time around.Permissions: Whenever webcat moves an orderfile, it changes the ownership of that file from the shared developer owner with mode 664 to owner www and the 644 permissions (no group write). User-created shoppingcarts are www/www and mode 664, which is fine until webcat authorizes them, after which they are www/www and mode 644. Our developers often need to manipulate these orderfiles manually for various reasons. The developers are in the www group, and the parent directories (Orders, Completed, etc) all have 770 permission.A couple solutions come to mind:- set up a crontab to chmod all files in these directories every minute (did this already, but it's a sledgehammer solution)- swap out all the [movefile] contexts with [shell]mv oldpath newpath[/shell] contexts to preserve the file settings- make developers login as www (same user as webcat process), but I'm not too hot about the idea of giving user www a valid shell for security reasons.- report this as a potential bug to SMSI and hope they fix it in a future releaseOn a semi-related note: OSX Jaguar client -> Jaguar client file permissions are an equal pain in the a$$. The default permission set through personal file sharing are 600, which of course prevents apache from serving the files unless you force all developers to login as www (which, again, is not desirable). I haven't tried 10.2 server yet, but that's an expensive solution considering the fraction of 10 Server features we would be using. I'm going to try NFS instead of AFP since all our developers are running OSX.Any thoughts on the above?-Dale-------------------------------------------------------------This message is sent to you because you are subscribed to the mailing list .To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to Web Archive of this list is at: http://search.smithmicro.com/ Dale LaFountain

DOWNLOAD WEBDNA NOW!

SMSI: databases corrupted on crash and permission issues on OSX

2002

Top Articles:

Related Readings: