1. Home
  2. Application Programming Interface
  3. Filter survey data using the API

Filter survey data using the API

Collecting data via large surveys can become onerous. Therefore, Survalyzer has several capabilities within the API to easily filter survey data. These include:

  • Row filtering
  • Column filtering
  • Use of metadata
  • Content compression

Row filtering

The API /publicapi/Survey/ReadSurveyData contains the property “Filter” which allows the following types of Filter:

  • General
    • Start date
    • End date
    • Language
    • Status
  • Panel fields
  • URL variables

Several filter conditions can be combined and are evaluated with an “and” operator. An example for a common filter expression is extracting complete responses collected from a specific date onwards.

"filter": [
		"filterType": "General",
		"filterOperator": "IsGreaterThan",
		"generalFilter": "StartDate",
		"value": "2019-01-01"
		"filterType": "General",
		"filterOperator": "IsEqualTo",
		"generalFilter": "Status",
		"value": 1 //Completed

For compatibility reasons, the properties “language” and “status” are still in place and operational. Combining the old and the new filter properties is, however, not supported.

Column Filtering

The property FieldsToDownload contains a list with column names. As the columns are dynamic for each survey, this shows filtering for questions with the codes q1, q2 and q3:

"fieldsToDownload" : ["q1","q2","q3"]

This could help to significantly reduce workload if chunky fields like “user agent” or “showanswers” are not required for further processing.

Turn off metadata

The Code Plan is a powerful instrument to understand the structure of data. However, for BI or analytics tools this is just a surplus overhead. Therefore, an application was introduced to turn off metadata. Here is an example of how it works:

"loadCodePlan": false

For large surveys with many questions, especially matrix questions, or those with many variables, this could again significantly reduce the data download.

Content compression

When the API returns only that data which is essential, this could still potentially be a large data set – particularly if several thousand responses have been collected. To address this, we introduced content compression to our APIs. To enable content compression, add the following header to the request:

Accept-Encoding: gzip

This triggers a mechanism which compresses the result using a zip algorithm. Statistically, the content can be reduced by 85% for json.

Reason for the format change compared to the legacy API GetRawDataAsJson

The GetRawDataAsJson had a smaller return syntax. Therefore, some customers were wondering why Survalyzer switched to a more chunky format. The reason was simple: the previous format used a dynamic properties approach which made it impossible to define a schema/contract. So the format looked different for each survey. Some of our customers had trouble with this approach, due to its two inherent disadvantages:

  • The contract could not be registered in a Web Application Firewall as it didn’t follow a scheme
  • It was not possible to generate strongly typed client classes because the result was an object like this: {}

With the new format, both of these issues are addressed. Client classes for most of the major programming languages can be generated on the webpage https://editor.swagger.io/.

Overall, the new format is even more efficient, in terms of data transfer and content selection.

Updated on August 26, 2019

Was this article helpful?

Related Articles