In our final project, we have built a multi-functional car controlled by a web application on the PC, which allows users to drive the car as well as play music from the interface on the website.
- Driving Mode: Move toward one specific direction
- Speaker Mode: Read .WAV files stored in the SD card and play it through speakers, as well as showcase the amplitude of the audio on the TFT display
- Piano Mode: Play certain notes of one piano and display which key being pressed on the TFT
In general, we applied what we have learned in this course into this project and integrating many functions together was quite fun and a good learning experience.
High level design
Our group decided to pursue this subject as our final project because we can not only leverage what we have learned through this semester, such as generating PWM to run motors by output compare units, playing music through DAC, serial communication through UART, concurrent programming and etc, but also learn a lot of new concepts such as file system, operations on shared SPI channels, waveform audio file format and some modern front-end technologies.
The audio format to be played is .WAV and it stores data in “chunks”. There are two sections in the .WAV files: the header and the data. The header is the beginning of a WAV(RIFF) file. The header is used to provide specifications on the file type, sample rate, sample size and bit size of the file, as well as its overall length. The header of a WAV(RIFF) file is 44 bytes long. The data section is the individual samples. An individual sample is the bit size times the number of channels.
In the header section, for example, bytes from 40 to 43 give the actual filesize. So the filesize is given as
The Timer3 interrupt is enabled to transmit audio data to the DAC based on the sample rate of the music, and it is also easy to get the sample rate of the music given by
As for the data section, for 16-bit PCM files, the data is stored as 16-bit Signed samples, whereas for 8-bit PCM files, the data is stored as 8-bit Unsigned samples. The way the audio data is origanized is shown below. Considering the MCP4822 has 12-bit resolution, the audio samples are converted to 12-bit values along with channel specifications for the DAC. For mono files, both channels play the same value. As for 8-bit samples, the 12-bit data is obtained by bit-shifting left 4 times. For 16-bit samples, the 12-bit data is obtained by combining the 8 bits of the higher byte with the 4 bits of the lower byte.
There are also two FIFO(First-In-First-Out) cyclic buffers for both channels and the size of these cyclic buffers is 2000. The cyclic buffers are used to ensure smooth audio playback and I managed two indices ReadIndex and WriteIndex to indicate the positions where to retrieve the audio data sent to the DAC and where to store the new audio data read from the SD card. Besides, considering the yield time for this reading thread is 10 milliseconds, it would be appropriate to read more than 0.01 * sample rate audio data into the cyclic buffer at the first time. And there are also two circumstances shown below to indicate how much data should be read from the SD card. When the ReadIndex is larger than the WriteIndex, the total number of data can be stored in the cyclic buffer is given by Buffer Size – (ReadIndex – WriteIndex). Otherwise, the total number is given as (WriteIndex – ReadIndex).
At a high level, our project can be divided into 6 components shown in Figure 3. The PIC32 is the heart of our project and it is used to control different sections. When the user send requests from the interface on the website, the runtime server Node.js would take over the command and send it to the serial port that is connected to one Bluetooth module. This information is communicated in real-time by the means of Bluetooth transmission, the PIC32 can receive that command through the serial communication with the other Bluetooth module and based on the command transmitted from the PC, it will perform different functions by controling different sections including generating PWM signals to run motors, reading audio data in the SD card, displaying different patterns on the TFT display or sending audio data to the DAC.
Both hardware and software significantly contributed to the success of the project. However, we had to make some trade-offs between hardware and software functionalities. For example, one of the design decisions we made is to use HM-10 as the Bluetooth module to implement the wireless communication between PIC32 and PC because it is quite reliable, but the downside of the HM-10 module is that we need to manually enter AT command to bind the two Bluetooth modules every time we turn on the whole circuit, which is not what we expected. But we successfully solved this in software and each time while starting the Node.js web server, it will automatically send the AT command to bind the Bluetooth modules first. Although there are some other Bluetooth modules that can store the bounded addresses and automatically bind with each other, the HM-10 is the most reliable module among all the Bluetooth modules we have ever tried. When we used HC-05, there are even some errors when sending commands to the PIC32 and we tested that by displaying sent commands on the TFT. For example, there was one time when we tested our HC-05 Bluetooth module and we transmit a character ‘w’ through serial monitor, but the PIC32 receives ‘t’ instead for unknown reasons.
Another design decision is that we decided to take the TFT off the Big Board. In our project, the TFT display and the MicroSD adapter shares the same SPI 1 channel. As shown in the course website, it is easy to discover that the MISO of TFT is left unconnected and the MOSI is connected to RB11 on the PIC. However, due to the hardware limitation of the Peripheral Pin Select(PPS): RB11 can be both SDO1 and SDI1 but RB13 can only be SDO1. So in order to ensure they can work properly sharing the same SPI channel, we took the TFT off the Big Board to make it share the SCK, MOSI, and MISO with the MicroSD adapter and also modified the tft library. At the same time, since the MicroSD adapter shares the same SPI channel with the TFT display, there is not much time left to draw complicated patterns on the TFT display and the SD card adapter will take up the SPI 1 channel for the most time, for which reason it only displayed some simple patterns on the TFT.
The Bluetooth standard (IEEE 802.15.1) is commonly used with devices that involve Bluetooth transmission of data. This standard states that Bluetooth should be used in a range of 100m with 1-3 Mbps data transmission, a bandwidth of 2.14 GHz, a power consumption from 2.5 to 100 mW and uses that include short range control and/or monitoring .There are no patents, copyrights, or trademark considerations relevant to this project.
As is shown in Figure 3, there are 5 peripheral components connected with the PIC32 and to accomplish this project, we decided to tackle smaller parts individually and assemble them at the end. The 5 peripheral components are as follows：
- Motor diver and motors(Output Compare Unit)
- TFT display (SPI1)
- MicroSD adapter (SPI1)
- DAC, amplifier and speakers (SPI2)
- Bluetooth (UART)
Since we have 4 motors in our project, so we need 2 L298N motor drivers and each L298N is connected with 2 motors. We use +7.4V to power the L298Ns. As shown in Figure 5, the mechanism of L298N is that there are two outputs in each side, which are connected to the positive and negative pole of the motors. The Enable A and Enable B are connected to two separate PWM outputs from PIC32, Enable A controls the speed of Motor A, and in the same way, Enable B controls the speed of Motor B. The logic pins Input 1, Input2 take charge of the working state of Motor A and Input3, Input4 control Motor B. The Input3 and Input4 work the same way as Input1 and Input2, as shown in Table 1. In our project, Enable A is connected to RB7, Enable B is connected to RB8.
The display we are using is a 320×240 color LCD with 16-bit color specification. Communication is by SPI 1 channel from the PIC32. The circuit diagram is shown in the Figure 6. The SCK is connected to RB14. The MISO and MOSI are connected to SDI(RB11) and SDO(RB13) of the SPI 1 channel, which shares with MicroSD card adapter. In addition, the Chip Select line is connected to RB1 and the RST is connected to RB2. The D/C port connects to RB0. Vin connect to the Vin on the Big Board. GND is the same GND as PIC32.
The circuit diagram is shown in Figure 7. Similar to the TFT, the MISO and MOSI are connected to SDI(RB11) and SDO(RB13) of the SPI 1 channel. The SCK is connected to RB14. However, the only difference between the connection with the TFT display is that the Chip Select line is connected to RB3 while the chip select line of TFT display is connected to RB1. Vcc connects to the Vin on the Big Board. GND is the same GND as PIC32.
MCP4822 is a dual channel 12-bit Digital-to-Analog converter (DAC) with internal voltage reference. This device offers high accuracy and low power consumption, and is available in various packages. Communication with the device is accomplished via a simple serial interface using SPI protocols. In our project, the communication is accomplished via SPI 2 channel.
PAM8403 is a powerful and small Class D amplifier mounted on a 28 x 20 mm breakout board. It includes a super smooth potentiometer combination volume / on-off knob.
In our project, the PIC32 writes the audio data with channel configurations to the SPI 2 channel, and the analog output signal will be amplified through a PAM8403 amplifier. In the end, it outputs to 2 speakers. The Chip Select line of DAC is connected to RB4, and the SCK is connected to RB15. Vin of DAC is +3.3V. The full connection is shown in Figure 8.
Wireless Communication with PC is accomplished through two HM-10 Bluetooth modules: one for PC and the other one for the PIC32. On the PIC32 side, the communication between the PIC32 and the Bluetooth module is accomplished by serial communication through UART. The RX of the Bluetooth module is connected to the TX of the PIC32, which is RB10, and the TX of the Bluetooth module is connected to RA1, which is the RX of the PIC32. GND is connected to the common ground and VCC is +5V on the Big Board.
At first, we need to bind the two Bluetooth modules. And we used a serial monitor to bind the Bluetooth modules. The steps are listed below:
- Set the arduino serial monitor terminator to ‘no line ending’
- Set the main module to peripherial mode with ‘AT+ROLE0’
- Use ‘AT+ADDR?’ to get BLE address
- Set the main module to manual connect mode using ‘AT+IMME1’
- Set the main module to Central mode with ‘AT+ROLE1’
- Use ‘AT+CONaddress’ to connect. e.g. ‘AT+CONA78DB2F1405AA’. If the connection is successful you should see ‘OK+CONNA’
- Make sure that the LEDs stop blinking
The software program has two major parts: the program on the PIC32 and the code on the PC to set up the web server using Node.js and to create one website for interacting with events and sending requests to the server. The program in the PIC32 can be decomposed into several threads. To best discuss the program, it is easiest to elaborate the structure of the program following the program’s thread structure.
- LED Thread
- Serial Communication Thread
- Retrieving audio data from MicroSD card adapter and drawing objects on the TFT display Thread
- Animation Thread
The yield time for this timer thread is 1 second to toggle one LED. This thread is for debugging purpose.
This thread keeps scanning the serial buffer, PT_term_buffer that is declared in pt_cornell_1_3_2.h. We used the PT_GetMachineBuffer() method to get the input string from the PT_term_buffer. And according to different working modes of the car, the PT_terminate_time is different. Because in driving and speaker modes, we want the car to stop immediately whenever the user releases the key. So the PT_terminate_time should be short enough that if there is no input characters in the PT_term_buffer, then the car should stop, which means the outputs of the output compare units should be smaller than the threshold. However, in piano mode, the intention is that once you press a key for a while, you probably don’t want to play the same note twice. In this case, PT_terminate_time should be large enough to eliminate this kind of situation.
We used switch cases to construct one state machine to decide which case the PIC32 should take. The state machine can be divided into two parts. At first, the car is in default mode, which is driving mode and speaker mode. And also, the car can change into the piano mode or change into driving mode from piano mode anytime when the user wants. If the current state is in default mode, we can control the car move forward by pressing ‘w’, and move backward by pressing ‘s’, turning left by pressing ‘a’ or turning right by pressing ‘d’. If we click a music to play on our website, the client application will send command to the web server, which will send the same command to the serial port. Through Bluetooth communication, this thread will receive this command and the desired file name to get the related parameters, open timer3 interruption, and set interruption period corresponding to the sample rate of the music. If we pause the music, Playenabled and startread is false. We can also skip songs and go back to the former song then Playenabled and startread will also be false, which means the process of reading audio data from the SD card and sending audio data to the DAC terminates. If the car is in piano mode, the thread will call DrawKeyBoard() to draw the piano keypad on the TFT. Then get the desired audio file name based on which key the user presses and read audio data from the file, as well as send the audio data to the DAC. The flow chart of this thread is shown in Figure 10.
This is an important thread. We use this thread to read in the desired WAV file stored in the SD card. We have a global file pointer fp to point at the desired file. We have a boolean start read to record if we switch to either music mode or piano mode, and a boolean variable Playenabled to help us judge if we are can play our music ( it is the first time to read, the audio buffers are empty, in this case we cannot write to the DAC directly).
If Playenabled is false, we read the given fp file pointer, read in 1400*blockAlign(relate to WAV file itself) bytes, process the bytes we read in, generate the data output to DAC. Different WAV file have different blockAlign. For 8-bit MONO file, blockAlign = 1, for 8-bit stereo file, blockAlign = 2, for 16-bit mono file blockAlign = 2, for 16-bit stereo file blockAlign=4. The 8 or 16 bit here is the sampledepth mentioned in the GetMusicInfo() function. Then we use a four loop to get each byte and do two steps(the step of the for loop should equals to blockAlign since the generation of DAC signal use blockAlign bytes) 1. generate the value to write to DAC, 2. draw rectangulars on TFT screen. If the WAV file is 16-bit, we use the current byte and the next byte, get the lowest 4 bit of the next byte as the highest 4 digit of the value we want to output to DAC and draw on TFT, get the highest 4 bit of the current byte as the lowest 4 digit of the value we want to output to DAC and draw on TFT(If the channels==2, then process 4 bytes at a time, generate data for separate channels, 2 bytes each channel, do the same procedure to generate value for each channel). If the WAV file is 8-bit, just use the lowest 4 bit as the highest 4 bit of the DAC value. If the WAV file is 8-bit and 2 channels, take the next lowest 4 bit as the highest 4 bit of the data of the other channel. After having the value for DAC, we need to write to DAC. Remember to plus the value with 2048. Each time we generate a byte, save it to LeftAudioData and RightAudioData(a part of memory we allocated, unsigned int arrays). Then scale the DAC value to TFT scale, draw a rectangular for each byte. Same procedure happens when the Playenable is true, but this time we will read the rest data, instead of the first 1400 bytes.
If we are in piano mode, the DAC signal generation is the same. The only thing is that we will draw a red straight line pointing to the key we pressed (the drawing of keyboard is done the time we switch to piano mode). Here we show the working flow in Figure 11.
For this part, we got a lot of insight from Syed Tahmid Mahbub’s project “PIC32 Based Audio Player with MicroSD Storage”. We learned how the whole reading process should be executed, and what should we do to import MDD library to our own project. Also, we learned how to process read-in WAV file data. We will list the link at the reference section.
This yield time of this thread is 0.1 second to see if there reaches the end of the playing function. If the boolean value of end is true, which means it reaches the end of the music playing, then it will call TFT_ECE() to display the initial “ECE 4760” pattern on the TFT display.