Re: [WebDNA] Hard-coded db write delay when running certain code?

This WebDNA talk-list message is from

2011

It keeps the original formatting. numero = 106224
interpreted = N
texte = WebDNA 7.0 will append to RAM and automatically flush to disk every 1000 =records; otherwise, it writes to RAM and the db file should not show =anything on disk, since this is the RAM copy which is updated. May be =your script adds 1000 records every 2min20s? Maybe you are appending to =a huge database or to a slow disk (if you checked the option flush to =disk after each write) and it takes 2min20s to be written to disk?Could you provide your code and the coordinates the server you =tcpconnect to and i will try this on my side.- chrisOn Jan 31, 2011, at 11:37, Kenneth Grome wrote:> I'm running a tcpconnect script to extract data from an online =database on a non-webdna server, and I'm seeing EXACTLY 2 minutes and 20 =seconds in between db writes.> I'm using a 10 second delay in a meta-refresh tag so the page is only =supposed to delay 10 seconds before reloading itself and sending the =next tcpconnect. This meta-refresh delay works fine, but something is =up with my db writes ...> The results that get written to my results.db each time a new =tcpconnect is processed is always exactly 2 minutes and 20 seconds later =than the last db write. Just take a look at these timestamps and you'll =see what I mean:> 01/31/2011@06:30:37> 01/31/2011@06:32:57> 01/31/2011@06:35:17> 01/31/2011@06:37:37> 01/31/2011@06:39:57> 01/31/2011@06:42:17> 01/31/2011@06:44:37> 01/31/2011@06:46:57> 01/31/2011@06:49:17> 01/31/2011@06:51:37> 01/31/2011@06:53:57> 01/31/2011@06:56:17> 01/31/2011@06:58:37> 01/31/2011@07:00:57> 01/31/2011@07:03:17> 01/31/2011@07:05:37> 01/31/2011@07:07:57> 01/31/2011@07:10:17> 01/31/2011@07:12:37> As you can see, every db write is exactly 2 minutes and 20 seconds =later than the previous one. To me this suggests that WebDNA's has some =kind of a "time delay" built into it, and that's why every one of these =records is written to the db on exactly the same schedule.> Does WebDNA 7.0 have some internal code that's preventing db writes =more frequently than 2:20?> I think there must be some code inside the WebDNA 7.0 software that is =delaying each write to the db until exactly 2:20 after the last write, =or at the very least I believe that WebDNA has some internal code that =is causing the results seen above. =20> Because it seems virtually impossible that these db writes would occur =on such a precise and predictable schedule otherwise, especially when =the data being written is obtained via tcpconnect which naturally =includes unpredictable response times from the remote server.> Instead of seeing these 2:20 delays for every db write I SHOULD be =seeing a variety of different times inbetween db writes -- unless =something internal to WebDNA is causing this pattern.> Here's my code structure:> =--------------------------------------------------------------------------=-------> [search db=3Dsearch.db ... &max=3D1]> [founditems]> [text]content=3D> [middle startafter=3Dxxx]> [tcpconnect]> [tcpsend]> [/tcpsend]> [/tcpconnect]> [/middle]> [/text]> [append =db=3Dresults.db]stamp=3D[date]@[time]&content=3D[content]...[/append]> > [/founditems]> [/search]> [flushdatabases]> =--------------------------------------------------------------------------=-------> Something internal to WebDNA *MUST* be controlling the timing of db =writes. That's my conclusion until someone tells me otherwise.> By the way, when I use Firefox to visit the page specified in my =tcpconnect I get a response in a couple of seconds, so I know for a fact =that the same pages I'm retrieving via tcpconnect should not take =anywhere near 2:20 to be returned. =20> But that's how long it's taking until thedata is written to the db, =and of course the response page cannot be returned until my WebDNA code =has been fully processed. This is why I think the delay is the result =of some internal WebDNA code that's preventing db writes until the "next =interval" -- which appears to be 2:20 since the last db write, or =perhaps on the next x seconds of time that ends in 17, 37 or 57?> Please if you know something about this, let me know what's going on =here. My script is working fine except for these frustrating and =unexplained db write delays, and it seems to me that given the precise =time interval of these db writes there must be a WebDNA explanation for =this behavior.> Thanks.> Sincerely,> Kenneth Grome> --------------------------------------------------------- This message =is sent to you because you are subscribed to the mailing list . To =unsubscribe, E-mail to: archives: =http://mail.webdna.us/list/talk@webdna.us Bug =Reporting:support@webdna.us Associated Messages, from the most recent to the oldest:

Re: [WebDNA] Hard-coded db write delay when running certain code? (christophe.billiottet@webdna.us 2011)
[WebDNA] Hard-coded db write delay when running certain code? (Kenneth Grome 2011)

WebDNA 7.0 will append to RAM and automatically flush to disk every 1000 =records; otherwise, it writes to RAM and the db file should not show =anything on disk, since this is the RAM copy which is updated. May be =your script adds 1000 records every 2min20s? Maybe you are appending to =a huge database or to a slow disk (if you checked the option flush to =disk after each write) and it takes 2min20s to be written to disk?Could you provide your code and the coordinates the server you =tcpconnect to and i will try this on my side.- chrisOn Jan 31, 2011, at 11:37, Kenneth Grome wrote:> I'm running a tcpconnect script to extract data from an online =database on a non-webdna server, and I'm seeing EXACTLY 2 minutes and 20 =seconds in between db writes.> I'm using a 10 second delay in a meta-refresh tag so the page is only =supposed to delay 10 seconds before reloading itself and sending the =next tcpconnect. This meta-refresh delay works fine, but something is =up with my db writes ...> The results that get written to my results.db each time a new =tcpconnect is processed is always exactly 2 minutes and 20 seconds later =than the last db write. Just take a look at these timestamps and you'll =see what I mean:> 01/31/2011@06:30:37> 01/31/2011@06:32:57> 01/31/2011@06:35:17> 01/31/2011@06:37:37> 01/31/2011@06:39:57> 01/31/2011@06:42:17> 01/31/2011@06:44:37> 01/31/2011@06:46:57> 01/31/2011@06:49:17> 01/31/2011@06:51:37> 01/31/2011@06:53:57> 01/31/2011@06:56:17> 01/31/2011@06:58:37> 01/31/2011@07:00:57> 01/31/2011@07:03:17> 01/31/2011@07:05:37> 01/31/2011@07:07:57> 01/31/2011@07:10:17> 01/31/2011@07:12:37> As you can see, every db write is exactly 2 minutes and 20 seconds =later than the previous one. To me this suggests that WebDNA's has some =kind of a "time delay" built into it, and that's why every one of these =records is written to the db on exactly the same schedule.> Does WebDNA 7.0 have some internal code that's preventing db writes =more frequently than 2:20?> I think there must be some code inside the WebDNA 7.0 software that is =delaying each write to the db until exactly 2:20 after the last write, =or at the very least I believe that WebDNA has some internal code that =is causing the results seen above. =20> Because it seems virtually impossible that these db writes would occur =on such a precise and predictable schedule otherwise, especially when =the data being written is obtained via tcpconnect which naturally =includes unpredictable response times from the remote server.> Instead of seeing these 2:20 delays for every db write I SHOULD be =seeing a variety of different times inbetween db writes -- unless =something internal to WebDNA is causing this pattern.> Here's my code structure:> =--------------------------------------------------------------------------=-------> [search db=3Dsearch.db ... &max=3D1]> [founditems]> [text]content=3D> [middle startafter=3Dxxx]> [tcpconnect]> [tcpsend]> [/tcpsend]> [/tcpconnect]> [/middle]> [/text]> [append =db=3Dresults.db]stamp=3D[date]@[time]&content=3D[content]...[/append]> [thisurl]">> [/founditems]> [/search]> [flushdatabases]> =--------------------------------------------------------------------------=-------> Something internal to WebDNA *MUST* be controlling the timing of db =writes. That's my conclusion until someone tells me otherwise.> By the way, when I use Firefox to visit the page specified in my =tcpconnect I get a response in a couple of seconds, so I know for a fact =that the same pages I'm retrieving via tcpconnect should not take =anywhere near 2:20 to be returned. =20> But that's how long it's taking until thedata is written to the db, =and of course the response page cannot be returned until my WebDNA code =has been fully processed. This is why I think the delay is the result =of some internal WebDNA code that's preventing db writes until the "next =interval" -- which appears to be 2:20 since the last db write, or =perhaps on the next x seconds of time that ends in 17, 37 or 57?> Please if you know something about this, let me know what's going on =here. My script is working fine except for these frustrating and =unexplained db write delays, and it seems to me that given the precise =time interval of these db writes there must be a WebDNA explanation for =this behavior.> Thanks.> Sincerely,> Kenneth Grome> --------------------------------------------------------- This message =is sent to you because you are subscribed to the mailing list . To =unsubscribe, E-mail to: archives: =http://mail.webdna.us/list/talk@webdna.us Bug =Reporting:support@webdna.us christophe.billiottet@webdna.us

DOWNLOAD WEBDNA NOW!

Re: [WebDNA] Hard-coded db write delay when running certain code?

2011

Top Articles:

Related Readings: