Re: [WebDNA] Math Precision Preshmilson

This WebDNA talk-list message is from

2019


It keeps the original formatting.
numero = 114956
interpreted = N
texte = 2584 My guess is this problem is due to the processor that uses = floating-point arithmetic, which means a rounding error can be = introduced in certain cases when given any fixed number of bits, a = calculation with real numbers will produce quantities that cannot be = exactly represented using that many bits. Therefore the result of a = floating-point calculation must often be rounded in order to fit back = into its finite representation. This rounding error is the = characteristic feature of floating-point computation. - chris > On Oct 31, 2019, at 01:24, Stuart Tremain = wrote: >=20 > That is very strange >=20 > I did a few versions with the same results on fcgi 8.5.1 >=20 > [math]5.4-6+0.0[/math] > [math]5.4-6+0.1[/math] > [math]5.4-6+0.2[/math] > [math]5.4-6+0.3[/math] > [math]5.4-6+0.4[/math] > [math]5.4-6+0.5[/math] =3D failed > [math]5.4-6+0.6[/math] =3D failed > [math]5.4-6+0.7[/math] > [math]5.4-6+0.8[/math] > [math]5.4-6+0.9[/math] >=20 >=20 > [math]5.0-6+0.6[/math] > [math]5.1-6+0.6[/math] > [math]5.2-6+0.6[/math] > [math]5.3-6+0.6[/math] > [math]5.4-6+0.6[/math] =3D failed > [math]5.5-6+0.6[/math] > [math]5.6-6+0.6[/math] > [math]5.7-6+0.6[/math] > [math]5.8-6+0.6[/math] > [math]5.9-6+0.6[/math] --------------------------------------------------------- This message is sent to you because you are subscribed to the mailing list talk@webdna.us To unsubscribe, E-mail to: talk-leave@webdna.us archives: http://www.webdna.us/page.dna?numero=3D55 Bug Reporting: support@webdna.us . Associated Messages, from the most recent to the oldest:

    
  1. Re: [WebDNA] Math Precision Preshmilson (christophe.billiottet@webdna.us 2019)
  2. Re: [WebDNA] Math Precision Preshmilson (Stuart Tremain 2019)
  3. [WebDNA] Math Precision Preshmilson (Michael Davis 2019)
2584 My guess is this problem is due to the processor that uses = floating-point arithmetic, which means a rounding error can be = introduced in certain cases when given any fixed number of bits, a = calculation with real numbers will produce quantities that cannot be = exactly represented using that many bits. Therefore the result of a = floating-point calculation must often be rounded in order to fit back = into its finite representation. This rounding error is the = characteristic feature of floating-point computation. - chris > On Oct 31, 2019, at 01:24, Stuart Tremain = wrote: >=20 > That is very strange >=20 > I did a few versions with the same results on fcgi 8.5.1 >=20 > [math]5.4-6+0.0[/math] > [math]5.4-6+0.1[/math] > [math]5.4-6+0.2[/math] > [math]5.4-6+0.3[/math] > [math]5.4-6+0.4[/math] > [math]5.4-6+0.5[/math] =3D failed > [math]5.4-6+0.6[/math] =3D failed > [math]5.4-6+0.7[/math] > [math]5.4-6+0.8[/math] > [math]5.4-6+0.9[/math] >=20 >=20 > [math]5.0-6+0.6[/math] > [math]5.1-6+0.6[/math] > [math]5.2-6+0.6[/math] > [math]5.3-6+0.6[/math] > [math]5.4-6+0.6[/math] =3D failed > [math]5.5-6+0.6[/math] > [math]5.6-6+0.6[/math] > [math]5.7-6+0.6[/math] > [math]5.8-6+0.6[/math] > [math]5.9-6+0.6[/math] --------------------------------------------------------- This message is sent to you because you are subscribed to the mailing list talk@webdna.us To unsubscribe, E-mail to: talk-leave@webdna.us archives: http://www.webdna.us/page.dna?numero=3D55 Bug Reporting: support@webdna.us . christophe.billiottet@webdna.us

DOWNLOAD WEBDNA NOW!

Top Articles:

Talk List

The WebDNA community talk-list is the best place to get some help: several hundred extremely proficient programmers with an excellent knowledge of WebDNA and an excellent spirit will deliver all the tips and tricks you can imagine...

Related Readings:

Spiders and Bots (2000) FAX solutions? (2001) WebCat2b13MacPlugIn - [showif][search][/showif] (1997) search / summary (2005) Multiple catalog databases and showcart (1997) Don't do this! (was: Crashing Type 3) (1999) Plain tex spacing for html? (2003) Setting up WebCatalog with Retail Pro data (1996) frames & carts (1997) Separate SSL Server (1997) Possible Bug in 2.0b15.acgi (1997) Quickie question on the email templates (1997) Happy Easter ... and grep string request ... (2004) delete a file? (1998) Help name our technology! (1997) webstar in demo (2000) Keep away (1997) Firesite and [referrer] atg broke (1997) Re[2]: 2nd WebCatalog2 Feature Request (1996) Problems searching from a FORM (1997)