GNU social JP
  • FAQ
  • Login
GNU social JPは日本のGNU socialサーバーです。
Usage/ToS/admin/test/Pleroma FE
  • Public

    • Public
    • Network
    • Groups
    • Featured
    • Popular
    • People

Embed Notice

HTML Code

Corresponding Notice

  1. Embed this notice
    David Chisnall (*Now with 50% more sarcasm!*) (david_chisnall@infosec.exchange)'s status on Saturday, 06-Dec-2025 16:22:57 JSTDavid Chisnall (*Now with 50% more sarcasm!*)David Chisnall (*Now with 50% more sarcasm!*)
    in reply to
    • myrmepropagandist

    @futurebird

    The TI calculator does what mathematics normally means (round to nearest).

    A programming language will do one of the things specified in IEEE 754 floating-point rounding modes. The default for Python (and C), I believe, is 'round towards zero', also known as truncation. This is implemented in floating point as a very simple normalisation operation (shift the mantissa so that the exponent is 1, which either shifts some zeroes in from the right or shifts some bits off to the right and loses them).

    The extra fun on the calculator might be that it's may be using some binary-coded decimal representation internally, because otherwise you have fun rounding errors from not being able to represent a lot of decimal fractions precisely in a binary floating-point value. More advanced calculators will store rational numbers fractions, which lets them avoid any precision loss until the final convert-to-decimal step that they do for display.

    In conversationabout a month ago from infosec.exchangepermalink
  • Help
  • About
  • FAQ
  • TOS
  • Privacy
  • Source
  • Version
  • Contact

GNU social JP is a social network, courtesy of GNU social JP管理人. It runs on GNU social, version 2.0.2-dev, available under the GNU Affero General Public License.

Creative Commons Attribution 3.0 All GNU social JP content and data are available under the Creative Commons Attribution 3.0 license.