1. Home
  2. Application Programming Interface
  3. Filter survey data using the API

Filter survey data using the API

Collecting data via large surveys can become onerous. Therefore, Survalyzer has several capabilities within the API to easily filter survey data. These include:

  • Row filtering
  • Column filtering
  • Use of metadata
  • Content compression

Row filtering

The APIs /publicapi/Survey/v1/ReadSurveyData and /publicapi/Survey/v1/ReadSurveyDataCompact contains the property “Filter” which allows the following types of Filter:

  • General
    • Start date
    • End date
    • Language
    • Status
  • Question variables
  • Panel variables
  • URL variables

Several filter conditions can be combined and are evaluated with an “and” operator. An example for a common filter expression is extracting complete responses collected from a specific date onwards.

"filter": [
	{
		"filterType": "General",
		"filterOperator": "IsGreaterThan",
		"generalFilter": "StartDate",
		"value": "2019-01-01"
	},
	{
		"filterType": "General",
		"filterOperator": "IsEqualTo",
		"generalFilter": "Status",
		"value": 1 //Completed
	}
]

For backward compatibility reasons, the properties “language” and “status” are still in place and operational. Combining the old and the new filter properties is, however, not supported.

Column Filtering

The property FieldsToDownload contains a list with column names. As the columns are dynamic for each survey, this shows filtering for questions with the codes q1, q2 and q3:

"fieldsToDownload" : ["q1","q2","q3"]

This could help to significantly reduce workload if chunky fields like “user agent” or “showanswers” are not required for further processing.

Turn off metadata

The Code Plan is a powerful instrument to understand the structure of data. However, for BI or analytics tools this is just a surplus overhead. Therefore, an application was introduced to turn off metadata. Here is an example of how it works:

"loadCodePlan": false

For large surveys with many questions, especially matrix questions, or those with many variables, this could again significantly reduce the data download.

Content compression

When the API returns only that data which is essential, this could still potentially be a large data set – particularly if several thousand responses have been collected. To address this, we introduced content compression to our APIs. To enable content compression, add the following header to the request:

Accept-Encoding: gzip

This triggers a mechanism which compresses the result using a zip algorithm. Statistically, the content can be reduced by 85% for json.

Two different methods for different needs

The ReadSurveyDataCompact method is the equivalent to the deprecated method GetRawDataAsJson. It allows to receive data in a dynamic javascript object form which doesn’t follow any fixed interface. This format proofed very handy in conjunction with BI tools and dashboards. Therefore, we reintroduced this method. The query interface is the same as for ReadSurveyData.

With both methods in place customers have the choice to use the right method for the given use case. Here are a few examples for the use of the different methods.

ReadSurveyData:

  • In Web Application Firewall (WAF) scenarios
  • If data is processed from a contract based system (Exporter)

ReadSurveyDataCompact:

  • In BI Tools like PowerBI or QlikView
  • In JavaScript Portals or Dashboards

Client Programming interfaces

Survalyzer offers customers a comfortable way to use the public API without caring about interfaces. Since the methods are following the OpenAPI Standard the client interfaces could be generated in all popular programming languages using the webpage https://editor.swagger.io/. Additional to this possibility Survalyzer maintains the client interface for .net based languages which could be downloaded here.

Updated on October 14, 2020

Was this article helpful?

Related Articles