The Roster Server preprocessor is an option for districts to adjust their data so that it is OneRoster compliant. Using the preprocessor, you are able to append users, enrollments, orgs, academic sessions, courses, and classes to existing files. You are also able to auto-generate enrollment dates, bulk modify usernames, set primary orgs for users, remove certain values in a column, and even populate columns with cell values derived from other columns.
To use the preprocessor, first head to the tab underneath Settings.
If you have never created a custom preprocessor config before, this page will be blank. To start, click on "Create New Config"
Set the name of your config so that you know what it does. You can also change the System Name and System Version.
If your data set has demographics and resources, check the boxes to include them in the import, otherwise Roster Server will not acknowledge them.
With the Manifest taken care of, head to Academic Sessions and set the schoolYear. If the school year ends in 2019, then put "2019" as the school year.
If you have old (expired) data you'd like to remove, type a value in the postfix settings. In this example, "2017" would remove academicSessions, classes, and enrollments from 2016 and earlier.
Moving on to Classes, you have the option of changing class titles so that they include either the teacher's name, or the course code and class title.
In Courses, you can add an orgSourcedId to each course if they are missing one. Generally, this should be the sourcedId of the district org.
In Demographics, you can change the default state to any of the 50 states.
In Enrollments, you are able to fill in missing enrollment dates by using the academic session of the associated class.
For example, a student enrollment of a class referencing the academic session beginning 2017-08-14 and ending 2018-06-03 will have a beginDate of 2017-08-14 and an endDate of 2018-06-03
The Orgs tab does not have any unique settings, but a common request is to add new orgs to Roster Server from a separate file. With the Append Files setting, you are able to attach any file or files to any existing OneRoster standard CSV. Just make sure the files are both spelled correctly, and have matching case. (This field is CASE-SENSITIVE)
Please be aware that the Append function will add the entries in the specified CSV to the main OneRoster CSV first before performing any checks or modifies. If an incorrect user entry exists in the main file, append will not overwrite the main file's mistaken entry and will throw an error for a duplicate entry.
In Users, you can automatically set each user as "enabled" and give them a default password.
You can also set a primary org for each user based on either the org appearing the most in the CSV file (usually the district org) or the first org that appears in the orgSourcedIds array.
Each tab in the preprocessor has these three actions which you can use to further adjust your roster data. Below are a few common actions used by our clients.
The transform action has a separate documentation page available here.
File Merge allows you to modify or add values to existing CSVs in your data set.
The File Name is the file that you want to merge with your base CSV. If you are looking to change roles within your users.csv file, a good name would be "changeRoles". Note that the ".csv" is automatically added to the end.
Field to Populate is the field in the base CSV which you want to change. In this case, it is the "role" field in users.csv.
Field to Read is the column in your merging CSV. This column contains the values you want to populate in your base CSV.
Source is the column name of the field you are using to match the two data sets together. For users, you have the option of matching by sourcedId, username, or identifier. Please keep in mind that File Merge is case-sensitive and will only merge files if the headers are spelled correctly and in the same case as the preprocessor. A merging CSV with a field header of "SOURCEDID" will not match with your base CSVs, even if the cell values are the same. The field header must be "sourcedId" or "username" or "identifier" with that exact casing.
Exclude allows you to remove certain rows from a particular CSV. In cases where there are users or orgs who you want to exclude from your data set, you can specify their sourcedIds here. Be aware: excluded entities will not transfer into Roster Server, and as a result, will not show up in the API.
Preprocessor action order:
It is important to know the order the various preprocessor actions are done in order to avoid one action overwriting the changes made by another. The order is as follows: file appends are done first, followed by transforms, then file merges, and finally postfixes. This means, for example, that a record that was transformed can potentially have that transformation overwritten by a file merge or postfix.
How to wipe all values in a given field:
How to remove the word "student" at the front of student sourcedIds:
How to remove the "@school.org" part of a user's username:
How to remove the district orgSourcedId from students with more than one orgSourcedId:
How to remove all admins: