Datetime precision loss

[jer_m]

I've noticed when I add Datetimes into a series there is a precision loss. Anything less then a millisecond seems to get thrown away. Here's an example of the problem using C# .Net

axTChart1.Series(0).XValues.DateTime = true;

DateTime dt = DateTime.Now;
double val = new Random().NextDouble();

axTChart1.Series(0).Clear();
axTChart1.Series(0).AddXY(dt.ToOADate(), val, "", 0);

double chartVal = axTChart1.Series(0).XValues.get_Value(0);
DateTime chartDT = DateTime.FromOADate(chartVal);
if (chartDT != dt)
MessageBox.Show("Not equal");

if (chartDT == new DateTime(dt.Year,dt.Month,dt.Day,dt.Hour,dt.Minute,dt.Second,dt.Millisecond))
MessageBox.Show("Equal");

Best regards,
Jerron

[Narcís]

Hi Jerron,

This is because dt was referencing Now method and its return value already changed at the moment of the comparision. However, the code below works fine.

Code: Select all

			axTChart1.Series(0).XValues.DateTime = true;

			double oadate = DateTime.Now.ToOADate();
			DateTime dt = DateTime.FromOADate(oadate);
			double val = new Random().NextDouble();

			axTChart1.Series(0).Clear();
			
			axTChart1.Series(0).ValueFormat = "";
			axTChart1.Series(0).AddXY(dt.ToOADate(), val, "", 0);
			
			double chartVal = axTChart1.Series(0).XValues.get_Value(0);
			DateTime chartDT = DateTime.FromOADate(chartVal);
			
			if (chartDT != dt)
				MessageBox.Show("Not equal");

			if (chartDT == new DateTime(dt.Year, dt.Month, dt.Day, dt.Hour, dt.Minute, dt.Second, dt.Millisecond))
				MessageBox.Show("Equal");

[jer_m]

Hi Narcis,

Thanks for the quick response,


Datetime is actually a structure, meaning it is a value type not a reference type.

DateTime dt = DateTime.Now;
causes a copy it doesn't set a reference, so dt never changes.



as an example, if DateTime where a class, making it a reference type then dt1 and dt2 should be equal.

DateTime dt1 = DateTime.Now;
System.Threading.Thread.Sleep(100);
DateTime dt2 = DateTime.Now;

if (dt1 != dt2)
MessageBox.Show("Not equal");



That said, this surprised me, DateTime.ToOADate() and DateTime.FromOADate() actually causes the loss of precision.

I don't know if I would say this is a bug, maybe this is just a limitation of the OADate format.

However, it does seem counterintuitive when the following doesn't always evaluate to be true
DateTime dt = DateTime.Now;
dt == DateTime.FromOADate(dt.ToOADate())



Even if this is a Microsoft bug, I know what the chances are of getting that changed I do have a few proposals for you though.

1. allow a way to pass in DateTimes into series (this would probably be a major headache)

2. add a flag in your series similar to
axTChart1.Series(0).XValues.DateTime = true;

that would look like this
axTChart1.Series(0).XValues.DateTimeAsTicks = true;

this way you can maintain reverse compatibility and instead of passing in the following
double oadate = d.ToOADate();

we could pass in (notice there would have to be a type cast somewhere, ticks is a long)
DateTime d = DateTime.Now;
double ticks = (double)d.Ticks;



the reason is there is no loss of precision when using Ticks
DateTime d = DateTime.Now;

long ticks = d.Ticks;
DateTime dt = new DateTime(ticks);

if (d == dt)
MessageBox.Show("Equal");


then you could internally do your conversions without a data loss, you would just have to know to construct your internal DateTime, or whatever your using, from either ticks or OADate's


Best regards,
Jerron

[Narcís]

Hi Jerron,

Thanks for your suggestions. I've added them to our wish-list to be reviewed and considered for inclusion in future releases.

[yves bourgeois]

Hi, Narcis,

we are experiencing the same problem as described in the original post. Do you have any information on whether the possibility of passing a datetime will be available soon?

kind regards

Yves Bourgeois

[Narcís]

Hi Yves,

Sorry but I can't give you an estimate date for this at the moment. I recommend you to be aware at this forum or subscribe to our RSS feed for new release announcements and what has been fixed on them.

[jer_m]

Hi Narcis,

Can you let me know the status of this issue?

Was this addressed at all in the newest release, 8.0.0.4

[Narcís]

Hi jer_m,

I'm afraid not, this hasn't been enhanced yet.

[Narcís]

Hi Jerron,

As an update to your previous request, we do not expect to add a modification to the ActiveX version for Tick support. TeeChart uses double values internally to handle DateTime and must use double precision when plotting. There may have been a possibility to advise a workaround by modifying the precision when dealing with datetime if it had been the case that date to double or float conversions could be correctly handled by the environment. Unfortunately it seems not possible to convert correctly a tick precision value to a double value.

Examples:

Works ok:

Code: Select all

      DateTime dt = DateTime.Now; 

      long newVal = dt.Ticks;
      DateTime newDT = new DateTime(newVal);
      if (newDT == dt)
        MessageBox.Show("equal"); 

try moving to double. Fails:

Code: Select all

      long tickValDouble = dt.Ticks;
      //no double overload for date ... make OADate
      //Chart and dateTime uses OADate double "/ 1E+17"
      double dateVal = tickValDouble / 1E+17;
      //lets convert back to check
      long checkVal = Convert.ToInt64(dateVal * 1E+17);
      if (tickValDouble != checkVal)
        MessageBox.Show("lost precision"); 

[jer_m]

Hi Narcís,

I see your point about the precision loss. However I’m assuming the code you used to convert between ticks and OLE Automation Dates was just to represent the data loss, here’s Microsoft’s actual conversion code via reflector

Code: Select all

        private static double TicksToOADate(long value)
        {
            if (value == 0L)
            {
                return 0.0;
            }
            if (value < 0xc92a69c000L)
            {
                value += 0x85103c0cb83c000L;
            }
            if (value < 0x6efdddaec64000L)
            {
                throw new OverflowException(Environment.GetResourceString("Arg_OleAutDateInvalid"));
            }
            long num = (value - 0x85103c0cb83c000L) / 0x2710L;
            if (num < 0L)
            {
                long num2 = num % 0x5265c00L;
                if (num2 != 0L)
                {
                    num -= (0x5265c00L + num2) * 2L;
                }
            }
            return (((double)num) / 86400000.0);
        }

static long DoubleDateToTicks(double value)
        {
            if ((value >= 2958466.0) || (value <= -657435.0))
            {
                throw new ArgumentException(Environment.GetResourceString("Arg_OleAutDateInvalid"));
            }
            long num = (long)((value * 86400000.0) + ((value >= 0.0) ? 0.5 : -0.5));
            if (num < 0L)
            {
                num -= (num % 0x5265c00L) * 2L;
            }
            num += 0x3680b5e1fc00L;
            if ((num < 0L) || (num >= 0x11efae44cb400L))
            {
                throw new ArgumentException(Environment.GetResourceString("Arg_OleAutDateScale"));
            }
            return (num * 0x2710L);
        }

After looking up the two different formats this is what I found

An OLE Automation date is implemented as a floating-point number whose value is the number of days from midnight, 30 December 1899. For example, midnight, 31 December 1899 is represented by 1.0; 6 A.M., 1 January 1900 is represented by 2.25; midnight, 29 December 1899 is represented by -1.0; and 6 A.M., 29 December 1899 is represented by -1.25.

DateTime.Ticks
The value of this property is the number of 100-nanosecond intervals that have elapsed since 12:00 A.M., January 1, 0001.

Given this you should be able to do something as follows

Code: Select all

DateTime dt = DateTime.Now;
//convert OLE Automation start date ‘midnight, 31 December 1899‘ to ticks = 0x85103C0CB83C000, then subtract that from our dt ticks which start at ‘12:00 A.M., January 1, 0001’
            long lng = dt.Ticks – 0x85103C0CB83C000L;
	//now we are on the same time base so we just have to convert from ticks ‘100nanoseconds to days’
//there are 86400000000000 nanoseconds in one day, so there are 864000000000 ticks in a day
double dbl = Convert.ToDouble(lng) / 864000000000d;

I didn’t have the time to thoroughly go through it all the way, but here’s what jumped out at me in Microsofts code.

Code: Select all

long num = (value - 0x85103c0cb83c000L) / 0x2710L;

or in decimal

Code: Select all

long num = (value - 599264352000000000) / 10000; 

the subtraction is setting the start date too OLE Automation start date, the division of 10000 I’m assuming is converting from ticks (100nanoseconds) to milliseconds. However, they set it to a long, which is why everything bellow a millisecond is lost.


When I get some more time I will mess around with converting to and from the DateTime and I will probably just end up doing it myself, However it would be nice to know, if I make this conversion and pass in the double, will tChart retain the precision when it’s used internally?

Best regards,
Jerron

[Narcís]

Hi Jerron,

When I get some more time I will mess around with converting to and from the DateTime and I will probably just end up doing it myself, However it would be nice to know, if I make this conversion and pass in the double, will tChart retain the precision when it’s used internally?

As a crosscheck we tried the routines TicksToOADate(long value) and with the same result that they lose precision (dropping more precision infact than the crude approach using "longTickVal / 1E+17"). Your following suggestions may give more positive results. Once the double arrives in TeeChart it will maintain precision to the following level:

Type Range Significant digits Size in bytes
Double 5.0 x 10^-324 .. 1.7 x 10^308 15-16 8

Meaning too, that what goes in should come out if comparing TeeChart Series values to what they were set as. I'm not sure how that will bear up to actually showing 100 nanosecond increments on the Chart, an approach for which an alternative would be to rescale nanosecond data mutiplying by (for example) 10xe6 or 9 and showing a Chart based on comparitives of 'like' numbers all at the same or similar precision level.