float or decimal – How to make an informed decision [C# .NET]

Hello World,

I encountered this issue while troubleshooting a bug reported by a test engineer and thought that it made a lot of sense to document my findings for reference. What follows is a description of the issue and what I learnt through my research.

The Issue

This application of mine shows a grid that contains data on some business parameters. The data is fetched from database, aggregated by the application and rendered as a tabular report. Now this report showed min, max and average of a dataset but oddly enough the table showed values that did not even exist in the database! My first reaction was to look at the code block that was doing the calculations – I suspected there may be some inadvertent rounding-off somewhere. The basic business requirement was to limit the figures shown to two places after the decimal point, and I had written the following extension method for doing this:

public static class DoubleExtensions
{
     public static double RoundDown(this double value, int digitsToRoundTo)
     {
         int factor = (int)Math.Pow(10, digitsToRoundTo);
         return Math.Truncate(value * factor) / factor;
     }
}

This code is inspired from this discussion on Strackoverflow.
Although the code is pretty small and self explanatory, there are some points to notice:

The overload of Math.Truncate being used has the signature

double Math.Truncate(double d)

The Gotcha!

Now, a less-than-obvious fact about double type in C# is that as per the specification of the language, float(System.Single) and double (System.Double) are binary floating point numbers as per IEEE standards. To be used when performance is desired with some level of inaccuracy tolerable. In simple words, when you assign a value of 2.8 to a variable of type double or float, it is being stored as a binary equivalent (base 2) of that value. When passing around the value as argument to a method or assigning it to the return value of a parameter, you may not always see 2.8 as expected (and this is by design).

If you care for the precision and have some tolerance for performance, use the base 10 type decimal. Notice the word “performance” – well, using the decimal type in code is not as fast as flot/double because to store a decimal value, there are additional steps required to convert it to a binary equivalent – remember, computers can only store zeroes and ones (and we thought they’re smart! 🙂 )

Our extension method then takes the following form:

public static class DecimalExtensions
 {
        public static decimal RoundDown(this decimal value, int digitsToRoundTo)
        {
            int factor = (int)Math.Pow(10, digitsToRoundTo);
            return Math.Truncate(value * factor) / factor;
        }
 }

Not such a big change after-all once you’ve figured it out 🙂 [Am I sounding like a manager or what!]

Further reading to get your head around the number types:

Expert’s voice: On Extreme Optimization
From the horse’s mouth: Jon Skeet on the data types Floating point, Decimal

Happy Coding!

Advertisements
Tagged with: , ,
Posted in .NET, C#
One comment on “float or decimal – How to make an informed decision [C# .NET]
  1. […] float or decimal – How to make an informed decision [C# .NET]. […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

  • Comic for March 28, 2017
    Dilbert readers - Please visit Dilbert.com to read this feature. Due to changes with our feeds, we are now making this RSS feed a link to Dilbert.com.
%d bloggers like this: