EEG Error Correction Interface

Serial:

For serial, we used the serial-USB cable. The UART receive pin (U2RX) was wired to RA1 on the board and the UART transmission pin (U2TX) was wired to RB10 on the board. Additionally, the serial-USB cable was grounded to the MCU ground, and the USB Vcc was not wired to anything.

Motor Control Hardware:

For the motor, we used a continuous rotation parallax servo. Since DC motors can cause inductive spikes that can blow out the transistors on the board, we used capacitors to reduce any chance of inductive spikes. We connected a 0.1 μF capacitor across the external 5V power supply and ground, and then a 1 nF capacitor across ground and the signal wire of the servo. The signal wire was connected to OC3, which is mapped to RPB9 (pin 18) on the board.

Game Play Hardware:

The game play hardware consisted of 3 buttons and 3 LEDs. The 3 buttons corresponded to start game (grey button), correct bin (blue button), and incorrect bin (red button). The start game button was wired to RA2 on the board and ground, the correct bin button was wired to RA3 on the board and ground, and the incorrect bin button was wired to RA4 on the board and ground. The LEDs corresponded to whether or not the player got the block in the correct bin as well as which bin was correct.

FFT Function:

For the FFT function, we used Bruce’s FFT algorithm from the FFT spectrum analyzer with TFT-LCD output. This function was optimized for use with fixed point numbers. We performed FFTs on arrays of size 256, so we defined N_WAVE as 256 and LOG2_N_WAVE as 8. This function returned an average of the magnitudes of all FFT bins in order to update our running average of baseline EEG activity which was used in other threads to determine anxiety spikes. To obtain this average, we added the power magnitude of all frequency bins together and divided by 128. The rest of this algorithm was consistent with Bruce’s implementation.

Change Direction Function:

This function set the duration of the PWM for the servo motor. The PWM signal was updated using this duration in the ISR. We changed the pulse width using a variable we called duration, and we used a switch statement using a variable called direction, where 0 corresponds to stop, 1 corresponds to counterclockwise rotation, and 2 corresponds to clockwise rotation. When direction was 0, the duration was set to 12000. When direction is 1, the duration was 12100, and when the direction was 2, the duration was set to 11900. These numbers were determined based on trial and error tuning of our specific servo motor.

Timer Thread:

The timer thread printed the time since reset at the top of the TFT. We kept track of time in order to determine when we believed we had enough baseline EEG data to begin playing the game (we typically had enough data about a minute after transmission began).

Motor Thread:

The motor thread controlled the motor and game play. First, we read the values from port A, bits 2, 3, and 4, which corresponded to the 3 buttons. The values read-in on these pins determined whether we were starting a trial, completing a trial when the correct outcome had occurred, or completing a trial when the incorrect outcome had occurred. We also saved the current state of the game and the previous state of the game, in order to determine whether we should complete actions that were only supposed to be carried out at the beginning of each round. When the start game button was low (meaning the button was pressed), we set valid_round high and score_incremented low. Score_incremented ensured that the score was only incremented one time per round even if the buttons were pressed more than once. We also printed “Start Game” so that the user would know that the game had started. Then, if either button_right or button_wrong went low at any point, we set valid_round equal to 0 to signal that the game had ended. We also increment the number_right or number_wrong based on which button was pressed and printed the updated scores. If button_right was pressed, we lit up an LED corresponding to the color of the collection bucket based on the direction that the conveyor belt was moving when the game completed. If button_wrong was pressed, we lit up a red LED in the middle of the conveyor belt to signal that the block landed in the incorrect bin. The next section of the thread checked if there was a valid round occurring. If a valid round was occuring, and it was the first cycle in this valid round (as determined by comparing valid_round to its previous value), we started the conveyor belt in a random direction from a randomly generated seed based on the current time. We compared our gamma value (from the pre-FFT data) to our running average from the FFTed raw data, as completed in the FFT thread. If the gamma value was at least 0.2 higher than the running average for 4 cycles, then we considered this an “anxiety event”. After an anxiety event was detected, it took 3 cycles of gamma data below the threshold for the debouncer to consider the EEG activity “not an anxiety event”. This ensured that the user had actually calmed down, instead of the PIC just receiving a transmission error. If there was not a valid game, meaning that the “correct” or “incorrect” end game buttons had been pressed, we set direction of the conveyor belt equal to zero and called change_direction(0) so the conveyor belt stopped. We also updated change, previous_change and the debounce counters to be ready for the next round.

Serial Thread:

The serial thread saved both the raw data and the pre-FFT data sent from the headband via the python script. The python script sent data every time it received an exclamation point. We used the thread PT_DMA_PutSerialBuffer, because this function used DMA for serial communication, which was fast. To use this thread, we spawned a child thread, which blocked the serial thread from executing while PT_DMA_PutSerialBuffer was executing. Next, we used PT_GetMachineBuffer to get the data from the python script. We used PT_GetMachineBuffer so that the data could be sent without a user inputting any data. We used a termination time of 5 seconds and a termination character count of 5 characters. If we received data, we scanned PT_term_buffer and saved the data value into cmd and its tag into a variable called value2. The tag indicated whether we were receiving raw data or pre-FFTed data, as well as which sensor the data was being received from in the case of raw data. If we do not receive anything, we set cmd to 0. Then, we divided cmd by 1000 and checked to see if it was less than 0.1 or greater than 0.999 to ensure that the values were in the correct range. If they were not in the correct range, meaning we received corrupted data, we set the value of cmd equal to its previous value. Though this caused us to lose some accuracy, we took this approach because EEG readings do not tend to change very quickly, and the timing of our system was slightly more important than making sure every data point was exactly correct. Then, we had a switch statement based on the value2 tag to save the data into the correct variables. Case ‘1’ was our gamma values (pre-FFTed values from the Muse headband), so we saved cmd into the variable ‘gam’. We ultimately decided to only save raw data from the two Muse sensors located on the forehead because these were the 2 main sensors associated with collecting data for gamma waves. Each of these sensors had two arrays. While one array was being filled, there was an FFT being performed on the other array. To ensure that we were filling the correct array, we use the variable ‘lock’ to indicate which array should be filled, and changed this value once the FFT had been performed. We also had counters for each sensor so that we ensured that we were filling in the correct index and that the array was full before we performed an FFT on it. Case ‘7’ was our AF7 sensor, so we saved the value into the respective array based on lock. If lock was equal to 1, this meant that there was an FFT being performed on F1af7 and F1af8, so we put cmd multiplied by the window into F2af7 at ctr3, which is the counter for the AF7 sensor. If lock was equal to 0, there was an FFT being performed on F2af7 and F2af8, so we put cmd multiplied by the window (initialized in main) into F1af7 at cr3. Lastly, we incremented ctr3 if it was less than 255, since the size of F1af7 andF2af7 was 256. Case ‘8’ was our AF8 sensor, so we saved the value into the respective array based on the lock. If lock was equal to 1, this meant that there was an FFT being performed on F1af7 and F1af8, so we put cmd multiplied by the window (initialized in main) into F2af8 at ctr4, which was the counter for the AF8 sensor. If lock was equal to 0, there was an FFT being performed on F2af7 and F2af8, so we put cmd multiplied by the window into F1af8 at ctr4. Lastly, we incremented ctr4 if it was less than 255, since the size of F1af8 and F2af8 was 256.

FFT Thread:

The FFT thread first checked if either set of raw data arrays contained 256 data points. If a set of arrays was full, it would call the FFT function on the filled arrays while allowing the other, unfilled arrays to be used in the serial thread to collect data. Once the FFT function was complete, all FFT bins were averaged together, and this average was used to update the running average baseline for anxiety thresholding. The running average was then printed at the bottom of the TFT display.

ISR:

The ISR changed the motor direction by adjusting the length of the PWM pulse. The desired motor direction was set in the Motor Thread.

Main:

In main, we initialized all of the threads and scheduled them based on round robin scheduling. We also initialized the motor and set the input pins. Additionally, we created the sine lookup table and the window array to help scale the data values to the correct range.

Python Script:

The python script first set up a server to receive the OSC stream from a phone. The phone was connected over bluetooth to the Muse Headband to receive both the raw data and the pre-FFT data. Once this server was established, methods were added to the server in order to receive data from specific transmission channels (i.e. in order to receive specifically from the raw sensor data channel or specifically from the gamma frequency data channel). These methods added the received data to predetermined queues, either the gamma queue or a raw sensor queue, given that the data actually contained a number as expected (instead of just garbage). Once the methods were added to the server, serial communication was established with the PIC32. We then used a multithreaded python script in order to receive OSC data and add it to queues, while simultaneously preprocessing and truncating some significant figures and sending this data with an appropriate tag over serial to the PIC32. We added character tags to all of our serial transmissions in case we could not guarantee the order of data transmission. For our analysis, it was important to identify whether we were sending pre-FFTed or raw data, as well as which sensor the raw data originated from.