1. Home
  2. Application Programming Interface
  3. Filter survey data using the API

Filter survey data using the API

Receiving data for surveys with a large amount of collected feedback could become painful to retrieve. Therefore, Survalyzer added several capabilities to the API to provide a comprehensive way of filtering survey data which are:

  • Row filtering
  • Column filtering
  • Turn of metadata
  • Content compression

Row filtering

The API /publicapi/Survey/ReadSurveyData contains the property “Filter” which allows the following types of Filter:

  • General
    • Start date
    • End date
    • Language
    • Status
  • Panel fields
  • URL variables

Several filter conditions can be combined and are evaluated with an “and” operator. An example for a common filter expression is getting completes beginning from a date.

"filter": [
	{
		"filterType": "General",
		"filterOperator": "IsGreaterThan",
		"generalFilter": "StartDate",
		"value": "2019-01-01"
	},
	{
		"filterType": "General",
		"filterOperator": "IsEqualTo",
		"generalFilter": "Status",
		"value": 1 //Completed
	}
]

For the sake of backward compatibility, the properties “language” and “status” are still in place and operational. Only combining the old and the new filter properties is not supported.

Column Filtering

The property FieldsToDownload contains a list with column names. Since the columns are dynamic for each survey the sample showing the filtering for questions with the codes q1, q2 and q3:

"fieldsToDownload" : ["q1","q2","q3"]

This could help to reduce the load massively if chunky fields like “user agent” or “showanswers” are not required for the further processing.

Turn off metadata

The Code Plan is a mighty instrument to understand the structure of the data. But for BI or analytics tools this is just overhead. Therefore, a property was introduced to turn off metadata. Here is an example how it works:

"loadCodePlan": false

For large surveys with many questions, especially matrix questions, or a lot of variables, this could reduce the data to download massively.

Content compression

When the API only returns the data which is really required, it could still be a lot especially if several thousand responses are collected. To address this, we introduced content compression to our APIs. To enable content compression, add the following header to the request:

Accept-Encoding: gzip

This triggers a mechanism which compresses the result using the zip algorithm. Statistically the content can be reduced by 85% for json.

Reason for the format change compared to the legacy API GetRawDataAsJson

The GetRawDataAsJson had a smaller return syntax. Therefore, some customers wondered why Survalyzer switched to a more chunky format. The reason is simple: The former format used a dynamic properties approach which makes it impossible to define a schema/contract for the format. The format looked different for each survey. Some of our large customers had trouble with this approach because it comes with two disadvantages:

  • The contract could not be registered in a Web Application Firewall since it doesn’t follow a scheme
  • It was not possible to generate strongly typed client classed because the result was only an object like this: {}

With the new format these two issues are addressed. Client classes for most of the major programming languages can be generated on the Webpage https://editor.swagger.io/.

With the above shown mechanisms and possibilities the new format could be even more efficient in terms of data transfer and content selection.

Updated on February 21, 2019

Was this article helpful?

Related Articles