Hi There How to truncate double type in an exact digit precision.
Hi There How to truncate double type in an exact digit precision.
05 Jan 2021, 08:00
HI there hope you guys doing well in this 2021 want some help here.
Edit : Problem Solved !!! but welcome to another suggestion and method. Thx.
( From Philippe Leybaert I just edit some )
----------------------
public double RoundDown(this double value, int digits)
{
double factor = Math.Pow(10,digits);
return Math.Truncate(value * factor) / factor;
}
---------------------
I want to compare double type with double type (which is a product price with product price or product price with some price level) and I found that we could use double.CompareTo() method.
but have some problem with that
assume
double a = 1.2345
double b = 1.23456789 if we compare these two on 4 digit precision it will get the equal result right.
so to use double.CompareTo() we need to get b price with an exact digit as a
I try to use Math.Round() but it doesn't give what I want Round() medthod it add the value of b up 1.2346 instead of 1.2345
is there a way then?
Thank you.
Replies
imrealfighter
07 Jan 2021, 03:04
RE: RE:
gmkenneyy said:
imrealfighter said:
HI there hope you guys doing well in this 2021 want some help here.
Edit : Problem Solved !!! but welcome to another suggestion and method. Thx.
( From Philippe Leybaert I just edit some )
----------------------
public double RoundDown(this double value, int digits)
{
double factor = Math.Pow(10,digits);return Math.Truncate(value * factor) / factor;
}---------------------
I want to compare double type with double type (which is a product price with product price or product price with some price level) and I found that we could use double.CompareTo() method.but have some problem with that
assume
double a = 1.2345
double b = 1.23456789 if we compare these two on 4 digit precision it will get the equal result right.
so to use double.CompareTo() we need to get b price with an exact digit as aI try to use Math.Round() but it doesn't give what I want Round() medthod it add the value of b up 1.2346 instead of 1.2345
is there a way then?
Thank you.Rounding double b = 1.23456789 to 4 decimal places will give you 1.2346 - so its correct!
Thank you.again gmkenneyy at first I thought that price on the chart is precisely calculated without round numbers but as result comeout yeap it uses Math.Round();
@imrealfighter
... Deleted by UFO ...