DownSampling

[sjhein]

I am writing a data processing application that can have as many as 15-20 series with 100000+ points per series. I am hoping to use the DownSampling function to help improve display performance when plotting these large data sets. I set up the DownSampling function using the example code from the "Feature Demo" example as a guide:

Code: Select all

Public Class Form1
    Private m_XValues1 As New List(Of Double)
    Private m_YValues1 As New List(Of Double)
    Private Sub Form1_Load(sender As Object, e As EventArgs) Handles Me.Load

        GetData()

        TChart1.Aspect.View3D = False
        TChart1.Legend.Visible = False
        Dim points As New Steema.TeeChart.Styles.Points()
        TChart1.Series.Add(points)
        Dim fastLine As New Steema.TeeChart.Styles.FastLine()
        TChart1.Series.Add(fastLine)

        Dim downSampling As New Steema.TeeChart.Functions.DownSampling(TChart1.Chart)
        points.Add(m_XValues1.ToArray, m_YValues1.ToArray)
        points.Active = False

        downSampling.DisplayedPointCount = 5000
        downSampling.Method = Steema.TeeChart.Functions.DownSamplingMethod.MinMaxFirstLastNull
        fastLine.TreatNulls = Steema.TeeChart.Styles.TreatNullsStyle.DoNotPaint
        fastLine.DataSource = points
        fastLine.Function = downSampling

    End Sub

    Private Sub GetData()
        Dim csv As String = "C:\Users\Public\Documents\downsample.csv"
        Using tfp As New FileIO.TextFieldParser(csv)
            tfp.SetDelimiters(","c)
            tfp.TextFieldType = FileIO.FieldType.Delimited

            Do Until tfp.EndOfData
                Dim fields() As String = tfp.ReadFields
                m_XValues1.Add(Double.Parse(fields(0)))
                m_YValues1.Add(Double.Parse(fields(1)))
            Loop
        End Using
    End Sub

    Private Sub TChart1_UndoneZoom(sender As Object, e As EventArgs) Handles TChart1.UndoneZoom
        TChart1(1).CheckDataSource()
    End Sub

    Private Sub TChart1_Zoomed(sender As Object, e As EventArgs) Handles TChart1.Zoomed
        TChart1(1).CheckDataSource()
    End Sub
End Class

My interpretation of this function is that by setting up the original source data in a separate series, whenever CheckDataSource is called, the original data would be re-downsampled to match the current chart dimensions. So, as you zoom in on smaller regions of the chart, you won't lose detail. When the number of source data points in the displayed region decreases below the DisplayedPointCount, I would expect the original source data to be displayed.

However, this isn't the behavior I am seeing and I want to determine if I have misinterpreted the design of this function or if I am missing some detail in setting it up (or something else is happening).

I tested this function using one of our actual data sets containing 100000+ points in a single series. Here is the full data set that has been downsampled to 5000 points. It appears to have been correctly downsampled and achieves our goals:

DownSampledFull.jpg (90.61 KiB) Viewed 12756 times

Here is the plot after zooming into the highlighted region above with downsampling enabled, and "CheckDataSource" called in the "Zoomed" event:

DownSampledZoomed.png (21.19 KiB) Viewed 12756 times

For comparison, here is the same region zoomed in to the original source data without downsampling:

OriginalZoomed.png (20.8 KiB) Viewed 12756 times

If you compare the original vs downsampled plot of the zoomed region, you can see that a great deal of detail has been lost in the downsampled plot. It appears to me that CheckDataSource is not re-calculating the downsampled plot to match the displayed region, it is simply displaying a zoomed version of the original 5000-point downsampling of the full data set.

I would appreciate any help in understanding what I am missing or misunderstanding here.

Thanks,
-Scott

[Marc]

Hello Scott,

We're checking this. on simple tests it seems ok, but I can see a problem when testing the new down-sample to be introduced with the next version, Douglas-Peucker. We'll do an overall check whilst we're reviewing that.

Regards,
Marc Meumann

[sjhein]

Marc,

Excellent - thank you.

-Scott

[Marc]

Hello Scott,

On closer examination the points plotted from DisplayedPointCount is not perfectly fixed according to total plotted points. There is a fixed tolerance that brings the display points down after a certain point. When tested on a 100,000 point series with a DisplayedPointCount at 1,000, the plotted function points held firm at that level until zooms went down to about 7,500 data points whereupon plotted downsampling points total begins to reduce. With this in mind, I would approach the problem in a different way perhaps; that when visible points are below a certain threshold[s], that you adjust the DisplayedPointCount accordingly or turn it off completely ( DisplayedPointCount = 0;).

Code: Select all

private void tChart1_Zoomed(object sender, EventArgs e)
{
  if (tChart1.Axes.Bottom.Maximum - tChart1.Axes.Bottom.Minimum < 5000)
		downSampling1.DisplayedPointCount = 0;
		line2.CheckDataSource();
}

You may find the new down-sampling method, Douglas-Peucker, available with the next update, a better fit for your needs. The reduction algorithm is better tuned to the data.

Regards,
Marc

[sjhein]

Marc,

I would approach the problem in a different way perhaps; that when visible points are below a certain threshold[s], that you adjust the DisplayedPointCount accordingly or turn it off completely ( DisplayedPointCount = 0;).

Thanks for the suggestion - I will investigate this alternative approach. I also look forward to evaluating the new down-sampling method when it becomes available.

-Scott