tag:blogger.com,1999:blog-82600244696369245392024-03-05T03:39:48.938-05:00Make yourself at OhmBrian Dwyerhttp://www.blogger.com/profile/09292143349696415766noreply@blogger.comBlogger13125tag:blogger.com,1999:blog-8260024469636924539.post-87889595432948113492017-03-15T15:22:00.001-04:002017-03-19T23:29:10.983-04:00Zero-Copy: CUDA, OpenCV and NVidia Jetson TK1: Part 2<div class="tr_bq">
In this part 2 post I want to illustrate the difference in technique between the common 'device copy' method and the 'unified memory' method which is more suitable for memory architectures such as NVidia's Tegra K1/X1 processors used on NVidia Jetson development kits. I wanted to show an example using just a CUDA kernel as well as an example utilizing OpenCV gpu::functions().</div>
<i><b><br /></b></i>
<br />
<h3>
<i>1. CUDA kernels: Device Copy method</i></h3>
<i><br /></i>
For this example, I've written a simple CUDA kernel that will take a fixed matrix (640x480) of depth values (delivered by Xbox 360's Kinect) and simultaneously convert to XYZ coordinates while rotating the points. This example only computes the Y dimension, but I can provide a full XYZ function as well, the math is fairly simple. The code may seem a bit intense, but try not to think of what's inside the CUDA kernel for now.<br />
<br />
Kernel Code:<br />
<br />
<span style="color: #444444; font-family: "courier new" , "courier" , monospace;">__global__ void cudaCalcXYZ_R2( float *dst, float *src, float *M, float heightCenter, float widthCenter, float scaleFactor, float minDistance)<br />{</span><br />
<blockquote class="tr_bq">
<span style="color: #444444; font-family: "courier new" , "courier" , monospace;"> //__shared__ float jFactor;<br /> __shared__ float shM[3];<br /> float nx,ny,nz, nzpminD, jFactor;<br /> int blockCapacity;<br /> int index;<br /> if(threadIdx.x == 0)<br /> {<br /> shM[0] = M[4];<br /> shM[1] = M[5];<br /> shM[2] = M[6];<br /> }<br /> index = blockIdx.x*blockDim.x + threadIdx.x;<br /> nz = src[index];<br /> jFactor = ((float)blockIdx.x - heightCenter)*scaleFactor;<br /> nzpminD = nz + minDistance;<br /> nx = ((float)threadIdx.x - widthCenter )*(nzpminD)*scaleFactor;<br /> ny = (jFactor)*(nzpminD);<br /> //Solve for only Y matrix (height vlaues)<br /> __syncthreads();<br /> dst[index] = nx*shM[0] + ny*shM[1] + nz*shM[2];</span></blockquote>
<span style="color: #444444; font-family: "courier new" , "courier" , monospace;">}</span><br />
<span style="color: #444444; font-family: "courier new" , "courier" , monospace;"><br /></span>
Basically, a float pointer is sent as <i>src</i> (Depth data), it is manipulated to acquire the 'Y' parameter which is then stored in another float* <i>dst</i>. In a device copy implementation of the CUDA kernel, the data pointed to by <i>src</i> must first be copied to device memory using CUDA method <i>cudaMemcpy</i>(). Below is an example of how to do this ('h' generally means host (cpu) while 'd' means 'device' (gpu) ):<br />
<br />
<span style="color: #444444; font-family: "courier new" , "courier" , monospace;">{<br />int rows = 480;<br />int cols = 640;<br />float* h_src, h_dst; //Host matrices<br />float* d_src, d_dst; //Device matrices<br />float* h_m, d_m; //4x4 rotation matrix (host/device)<br /><br />//Allocate device copies using cudaMalloc<br />cudaMalloc( (void **)&d_src, sizeof(float)*rows*480);<br />cudaMalloc( (void **)&d_dst, sizeof(float)*rows*480);<br />cudaMalloc( (void **)&d_m, sizeof(float)*16);<br /><br />//Allocate host pointers<br /> h_src = (float*)malloc(sizeof(float)*rows*cols);<br /> h_dst = (float*)malloc(sizeof(float)*rows*cols); </span><br />
<div>
<span style="color: #444444; font-family: "courier new" , "courier" , monospace;">h_m = (float*)malloc(sizeof(float)*4*4);<br /><br /> //Copy all matrices from host to device<br /> cudaMemcpy( d_src, h_src, sizeof(float)*rows*cols, cudaMemcpyHostToDevice);<br /> cudaMemcpy( d_m, h_m, sizeof(float)*16, cudaMemcpyHostToDevice);<br /><br />//Run the kernel<br /> cudaCalcXYZ_R2<<< rows , cols>>>(d_dst, d_src, d_m, 240, 320, 0.0021, -10);<br /><br />//Wait for GPU to finish<br /> cudaDeviceSynchronize();<br /><br />//Copy the result back to host memory<br /> cudaMemcpy( h_dst, d_dst, sizeof(float)*rows*cols, cudaMemcpyDeviceToHost);<br /><br />}</span><br />
<div>
<br />
<h3>
<i>2. CUDA kernels: Unified Memory method</i></h3>
<br />
Here we are going to utilize the same kernel as the above example, but this time we are going to avoid any memory copy altogether by utilizing the CUDA_UVA technique. Here, instead of using <i>cudaMallac</i>() we have to use <i>cudaMallocManaged</i>();<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">{<br />cudaSetDeviceFlags(cudaDeviceMapHost); //Support for mapped pinned allocations<br /><br /><span style="color: #444444;">int rows = 480;</span><br style="color: #444444;" /><span style="color: #444444;">int cols = 640;</span><br style="color: #444444;" /><span style="color: #444444;">float* h_src, h_dst; //Src and Dst matrices</span><br style="color: #444444;" /><span style="color: #444444;">float* h_m; //4x4 rotation matrix</span></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"><span style="color: #444444;"><br /></span>//Allocate float*s for CUDA. No need to allocate host and device separately<br /> cudaMallocManaged(&h_src, sizeof(float)*ros*</span><span style="color: #444444; font-family: "courier new" , "courier" , monospace;">cols</span><span style="font-family: "courier new" , "courier" , monospace;">);<br /> cudaMallocManaged(&h_M, sizeof(float)*4*4);<br />cudaMallocManaged(&h_dst, sizeof(float)*</span><span style="font-family: "courier new" , "courier" , monospace;">ros</span><span style="font-family: "courier new" , "courier" , monospace;">*</span><span style="color: #444444; font-family: "courier new" , "courier" , monospace;">cols</span><span style="font-family: "courier new" , "courier" , monospace;">);</span><br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">//Run the kernel</span></div>
<div>
<span style="color: #444444; font-family: "courier new" , "courier" , monospace;">cudaCalcXYZ_R2<<< rows , cols>>>(</span><span style="font-family: "courier new" , "courier" , monospace;">h_dst</span><span style="color: #444444; font-family: "courier new" , "courier" , monospace;">, </span><span style="font-family: "courier new" , "courier" , monospace;">h_src</span><span style="color: #444444; font-family: "courier new" , "courier" , monospace;">, </span><span style="font-family: "courier new" , "courier" , monospace;">h_m</span><span style="color: #444444; font-family: "courier new" , "courier" , monospace;">, 240, 320, 0.0021, -10);</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"><br />//Wait for GPU to finish<br /> cudaDeviceSynchronize();</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">//Done, now h_dst contains the results</span><span style="font-family: "courier new" , "courier" , monospace;">}</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">}</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: inherit;">So now we have completely eliminated copying over <span style="font-family: inherit;">1.23</span>MB (640x480x4<span style="font-family: inherit;"> </span>bytes) prior to running the kernel as well as eliminated copying 1.<span style="font-family: inherit;">23</span>MB (640x480x4 bytes) after the kernel has finished. Imagine trying to achieve real-time performance on a robot reading a Kinect sensor at 30FPS, needlessly copying more than <span style="font-family: inherit;">73.3MB</span> a second into the same RAM!</span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;"><b>[Note]:</b> This code would only function on an architecture such as the NVidia Tegra X/K processors, so no sense in trying to run it on your discrete GPU in your laptop or desktop (it just won't work!).</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<br />
<h3>
<i>3. OpenCV GPU functions: Device Copy method</i></h3>
There is a module available with OpenCV called GPU written in CUDA, for those to take advantage of GPU acceleration of various functions. There is plenty of documentation online to understand how to use OpenCV's CUDA, I will go over the very basics. The example we will use is the per-element multiplication of two matrices <i>a</i> and<i> b</i>, where the result is stored in <i>c</i>. Using the 'device copy' method, here is how to do so with OpenCV's gpu function <i>gpu::multiply()</i>:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">{</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">//variables/pointers</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">int rows = 480;<br />int cols = 640;</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">float* h_a, h_b, h_c;</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">float* d_a, d_b, d_c;</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;">//Allocate memory for host pointers</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">h_a = (float*)malloc(sizeof(float)*rows*cols);</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">h_b = (float*)malloc(sizeof(float)*rows*cols);</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">h_c = (float*)malloc(sizeof(float)*rows*cols);</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;">//Allocate memory for device pointers</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">cudaMalloc( (void **)&d_a, sizeof(float)*rows*cols);</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">cudaMalloc( (void **)&d_b, sizeof(float)*rows*cols);</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">cudaMalloc( (void **)&d_c, sizeof(float)*rows*cols);</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;">//Mats (declaring them using available pointers)</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">Mat hmat_a(cvSize(cols, rows), CV_32F, h_a);</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">Mat hmat_b(cvSize(cols, rows), CV_32F, h_b);</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">Mat hmat_c(cvSize(cols, rows), CV_32F, h_c);</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;">//Gpu Mats (declaring with available pointers)</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">gpu::GpuMat dmat_a(cvSize(cols, rows), CV_32F, d_a);</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">gpu::GpuMat dmat_b(cvSize(cols, rows), CV_32F, d_b);</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">gpu::GpuMat dmat_c(cvSize(cols, rows), CV_32F, d_c);</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;"></span><span style="font-family: "courier new" , "courier" , monospace;">//Let's assume our host matrices are filled with actual data, then copy them to the device matrices</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">dmat_a.upload(hmat_a);</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">dmat_b.upload(hmat_b);</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;">//Run gpu::multiply()</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">gpu::multiply(dmat_a, dmat_b, dmat_c);</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;">//Copy the result back to the host</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">dmat_c.download(hmat_c);</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;">//Result now in hmat_c, required copying matrix a, b and c...</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">}</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<br />
<h3>
<i>4. OpenCV GPU functions: Unified Memory method</i></h3>
<div>
<i><br /></i></div>
You'll notice that in the above example I've been allocating memory to pointers for my images, rather than just using OpenCV to allocate memory upon declaration of a Mat or GpuMat. This is required for this section on utilizing OpenCV GpuMats without having to upload and download data to and from the GPU memory on chips such as the Jetson IC. There is another less obvious reason I use this method. For real-time performance on embedded processors, it is more efficient to allocate memory for objects early on prior to any operations that run cyclically. As long as you can spare the memory, this becomes an effective way to increase performance (granted the trade off is sacrificing some RAM which won't be freed up etc.). If you find yourself in need of dynamically freeing up space from these allocated methods, you can look into <i>cudaFree</i>() and cudaFreeHost().<br />
<br />
Now on eliminating download() and upload() OpenCV function calls.<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">{</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">cudaSetDeviceFlags(cudaDeviceMapHost); //Support for mapped pinned allocations</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;">//variables/pointers</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">int rows = 480;<br />int cols = 640;</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">float* h_a, h_b, h_c;</span><br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">//Allocate memory for device pointers</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">cudaMallocManaged(&</span><span style="font-family: "courier new" , "courier" , monospace;">h_a</span><span style="font-family: "courier new" , "courier" , monospace;">, sizeof(</span><span style="font-family: "courier new" , "courier" , monospace;">float</span><span style="font-family: "courier new" , "courier" , monospace;">)*rows*cols);</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><span style="font-family: "courier new" , "courier" , monospace;">cudaMallocManaged(&</span>h_b<span style="font-family: "courier new" , "courier" , monospace;">, sizeof(</span>float<span style="font-family: "courier new" , "courier" , monospace;">)*rows*cols);</span></span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><span style="font-family: "courier new" , "courier" , monospace;">cudaMallocManaged(&</span>h_c<span style="font-family: "courier new" , "courier" , monospace;">, sizeof(</span>float<span style="font-family: "courier new" , "courier" , monospace;">)*rows*cols);</span></span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span><span style="font-family: "courier new" , "courier" , monospace;">//Mats (declaring them using pointers)</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">Mat hmat_a(cvSize(cols, rows), CV_32F, h_a);</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">Mat hmat_b(cvSize(cols, rows), CV_32F, h_b);</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">Mat hmat_c(cvSize(cols, rows), CV_32F, h_c);</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span><span style="font-family: "courier new" , "courier" , monospace;">//Gpu Mats (declaring with the same pointers!)</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">gpu::GpuMat dmat_a(cvSize(cols, rows), CV_32F, h_a);</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">gpu::GpuMat dmat_b(cvSize(cols, rows), CV_32F, h_b);</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">gpu::GpuMat dmat_c(cvSize(cols, rows), CV_32F, h_c);</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span><span style="font-family: "courier new" , "courier" , monospace;">//Run gpu::multiply()</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">gpu::multiply(dmat_a, dmat_b, dmat_c);</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span><span style="font-family: "courier new" , "courier" , monospace;">//Result now in hmat_c, no copying required!</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">}</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
Much like in the CUDA unified memory example, this method will only function on hardware with unified memory architecture (Jetson ICs for example). Now you do not need to bother using OpenCV download and upload methods for your algorithms.<br />
<br />
Enjoy the speedups!</div>
</div>
Brian Dwyerhttp://www.blogger.com/profile/09292143349696415766noreply@blogger.com10tag:blogger.com,1999:blog-8260024469636924539.post-26827771521867139672017-03-08T23:01:00.003-05:002017-03-15T22:04:57.238-04:00Zero-Copy: CUDA, OpenCV and NVidia Jetson TK1: Part 1If you aren't yet familiar with NVidia's embedded ECU releases (NVidia Jetson TK1, TX1 and coming soon TX2) they are definitely something dig into. NVidia has been embedding their GPU architectures on the same IC as decent-speed processors (like a quad-core ARM Cortex A-15). The Jetson TK1 is by far the most affordable ($100-200), and is an excellent option to bring in some high performance computing to your mobile robotics projects. I'm posting my findings on a slight difference between programming with CUDA on NVidia discrete GPUs and on NVidia's embedded TK/TX platforms (in regards to their memory architecture)<br />
<br />
CUDA is a C-based framework developed by NVidia to allow developers to write code for parallel processing using NVidia's GPUs. Typically the main CPU is considered the 'Host' while the GPU is considered the 'Device'. The general flow for using the GPU for general purpose computing is as follows:<br />
<ol>
<li>CPU: Transfers data from host-memory to device-memory</li>
<li>CPU: Command CUDA process to run on GPU</li>
<li>CPU: Either do other work or block (waiting) until the GPU has finished</li>
<li>CPU: Transfer data from device-memory to host-memory </li>
</ol>
This is the general flow, and you can read up on this more in depth. As far as the GPU and the CPU on the NVidia TK/TX SOCs go they are considered to have a Unified Memory Architecture (UMA): meaning they share RAM. Typical discrete GPU cards have their own RAM, thus the necessity of copying data to and from the device. When I learned this I figured the memory-copy process could be eliminated altogether on the Jetson!<br />
<br />
I started out learning how to do this purely with simple CUDA kernels. Generally, the CUDA compiler will not allow a CUDA function (or kernel) to operate on data-types that have CPU-type pointers etc. I came across different memory-methods of using CUDA:<br />
<ul>
<li>CUDA Device Copy</li>
<li>CUDA Zero Copy</li>
<li>CUDA UVA (Unified Memory)</li>
</ul>
It took me a while to get used to using these different techniques, and I was not sure which one was appropriate so I did some profiling in order to find out which one gave me the best speed-up (Device Copy as the base). It turned out that CUDA UVA was the better method for coding on the Jetson TK1 embedded GPU.<br />
<br />
However, I still ran into a problem using OpenCV on the Jetson. OpenCV has a CUDA module, however OpenCV is designed to use two different Mat data-types: mat for CPU and gpu::GpuMat for GPU. So you could not use OpenCV gpu::functions on cpu mat objects. OpenCV actually has you do the same thing as in 'device copy' for CUDA, and use their methods for copying a CPU mat to the GPU and vice-versa. When I realized this, I was stunned that there was no Unified Memory method (to my knowledge) in OpenCV. So all OpenCV gpu::functions required needless memory copying on the Jetson! On an embedded device this is an extreme bottleneck, as I was already hitting the wall with my programs working with the Kinect IR sensor and image data.<br />
<br />
So after quite a bit of sand-box style experimentation, I found the correct approach to casting Mat pointers into GpuMat pointers without doing any memory copy and maintaining the CUDA UVA style. My original program with my Kinect sensor ran at 7-10FPS, and that was with cutting the width and height down from 640x480 to 320x240. With my new approach of avoiding any memory copy I was able to achieve full 30FPS at full 640x480 (this is on all the Depth Data from the IR sensor).<br />
<br />
I will post code on my github and update this with the link soon.<br />
<br />
<a href="http://ohmwardbond.blogspot.com/2017/03/zero-copy-cuda-opencv-and-nvidia-jetson_15.html">Move on to Part 2 for examples</a>Brian Dwyerhttp://www.blogger.com/profile/09292143349696415766noreply@blogger.com1tag:blogger.com,1999:blog-8260024469636924539.post-86056076788005350232017-03-06T10:30:00.002-05:002017-03-10T15:55:03.150-05:00Ethernet-Based IMU for ROS<div class="Textbody">
So I've been doing a bit of development with Robot Operating
System (ROS) and a while back needed a flexible solution for acquiring inertial
measurement data from multiple sensors. My intention was to have an embedded
controller responsible for collecting IMU data and delivering to a master
controller. Since in ROS it is easy to swap between a laptop and an
embedded-Linux device as your platform, I decided to make a portable Ethernet-based
IMU out of the MPU9150 and a raspberry pi (or pcduino).<o:p></o:p></div>
<div class="Textbody">
<br /></div>
<div class="Textbody">
I had previously made my own C/C++ Linux-based API interface
for communicating to the MPU9150 (3axis gyrometer, accelerometer,
magnetometer). This project is an extension to have an embedded server which would send
periodic packets containing latest IMU data (so as of now, the embedded-IMU
device does not run ROS). I wrote a ros-node to connect to this
server, receive the data and publish it as a ROS message. Everything seems to be
working out quite well and I currently am receiving IMU data at 100Hz using
Ethernet (less over wifi). I can add additional Ethernet-based IMUs to the
project with little complexity now. Below is my wireless version.<o:p></o:p></div>
<div class="Textbody">
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiFjyxhTQVhhjy2maTvP-7YttFNn9SwAVmNEA9l7anNvc6FFzFhGHD9ZT5XBLaJt83BIpPegdLcs3VqPyCYf7G-L_kZ8jkdAlIE1WftKm0tZbXY_lX9CqbMEagALaKPtg06kKbMDMftgJ3-/s1600/imu_battery.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto; text-align: center;"><img border="0" height="384" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiFjyxhTQVhhjy2maTvP-7YttFNn9SwAVmNEA9l7anNvc6FFzFhGHD9ZT5XBLaJt83BIpPegdLcs3VqPyCYf7G-L_kZ8jkdAlIE1WftKm0tZbXY_lX9CqbMEagALaKPtg06kKbMDMftgJ3-/s640/imu_battery.jpg" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Pcduino-3, MPU9150 and a USB battery pack</td></tr>
</tbody></table>
<div class="Textbody">
<i style="text-align: center;"><br /></i></div>
<div class="Textbody">
For show, I setup a battery-powered pcduino-3 connected to an
IMU as the server over wifi (eliminating wires which get in the way for a
hand-held demo). On my ROS device (laptop) I have a node running which is
dedicated to receiving the IMU data packets and publishing in ROS as a sensor message that other ROS nodes can subscribe to. Below is a video of real-time plotting of
received IMU data using rqt. The video is not the best quality because my phone is not so great, but you can see that the top plot is the Gyro-Z measurement and the bottom plot is the Accelerometer x-axis measurement.</div>
<div class="Textbody">
<o:p></o:p></div>
<div class="Textbody">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<iframe allowfullscreen="" class="YOUTUBE-iframe-video" data-thumbnail-src="https://i.ytimg.com/vi/odOY1IQ3FBI/0.jpg" frameborder="0" height="266" src="https://www.youtube.com/embed/odOY1IQ3FBI?feature=player_embedded" width="320"></iframe></div>
<div class="Textbody" style="text-align: center;">
<o:p><a href="https://youtu.be/odOY1IQ3FBI" target="_blank">Plotting Wireless IMU in ROS </a></o:p></div>
<div class="Textbody">
<o:p><br /></o:p></div>
<div class="Textbody">
I also wrote an additional node in ROS to subscribe to the
IMU messages and perform some integration on the gyro data to estimate the
sensor orientation around the z-axis (so what a compass would tell you). You
can see in this short video the integration is somewhat reliable. There is definitely gyro drift over time
which would end up corrupting the estimate in the long term, but that's where filtering techniques will come in to assist in state estimation (for example: Kalman
filtering). Video below:<o:p></o:p></div>
<div class="Textbody">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div style="text-align: center;">
<iframe allowfullscreen="" class="YOUTUBE-iframe-video" data-thumbnail-src="https://i.ytimg.com/vi/xIeBdFcI8ak/0.jpg" frameborder="0" height="266" src="https://www.youtube.com/embed/xIeBdFcI8ak?feature=player_embedded" width="320"></iframe></div>
<div class="Textbody" style="text-align: center;">
<a href="https://youtu.be/xIeBdFcI8ak" target="_blank">Heading Estimate Using Gyro with Wireless IMU in ROS </a></div>
<div class="Textbody" style="text-align: center;">
<br /></div>
<div class="Textbody" style="text-align: left;">
I will be posting my code for this project on my github soon, I will update this once that is ready (and hopefully making better videos)</div>
<br />
<div class="Standard">
<br /></div>
Brian Dwyerhttp://www.blogger.com/profile/09292143349696415766noreply@blogger.com0tag:blogger.com,1999:blog-8260024469636924539.post-12404980960734644572014-12-10T21:43:00.004-05:002014-12-11T12:11:40.069-05:00Self-Balancing RobotIn my last class for my Masters, I decided to build a self-balancing robot for the final project. What I liked about this project, is that it involved a few very relevant areas of embedded systems all in one project. I'll try to write this post the way I wrote the presentation.<br />
<br />
The class was 'Mixed Signal Embedded Systems', and revolved around the Cypress chip called the PSoC4 (programmable-system-on-chip). The chip is interesting in that it has both an ARM M0 processor AND some (emphasis on 'some') programmable logic (PLDs). It also has 2 embedded amplifiers and some other interesting HW components (all configurable through the ARM via registers). Anyway, this post is not about this chip, but feel free to read up on it. It is another example of the direction in which embedded systems are heading.<br />
<br />
<h3>
<b>Motivations for this project</b></h3>
<ul>
<li>Interface to inertial measurement sensors (Gyros/Accelerometers)</li>
<li>Employ light-duty sensor fusion for robot pose estimation</li>
<li>Implement an embedded PID controller to maintain robot balance</li>
</ul>
<br />
<ul>
</ul>
<h3>
<b>The problem: The Inverted Pendulum</b></h3>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhz4XX4-NxMVSFhpChidit0QBNGBLdl3WtuNy8-PEs6Y4yCuh6U6vtvPSBxgouYRdmltLZOc-F23y_L_mDavEtFyRmpy1YmjjupTWbySGWxO2VpIln1FrdkLxCswmW0sOQ1nTvxL6KsEObW/s1600/pendulum.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhz4XX4-NxMVSFhpChidit0QBNGBLdl3WtuNy8-PEs6Y4yCuh6U6vtvPSBxgouYRdmltLZOc-F23y_L_mDavEtFyRmpy1YmjjupTWbySGWxO2VpIln1FrdkLxCswmW0sOQ1nTvxL6KsEObW/s1600/pendulum.png" height="298" width="320" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
The goal of the inverted pendulum is to keep the pendulum from tipping over. Without intervention, the pendulum is naturally unstable and will eventually tip over. Controls must be used to counter act this tendency by applying a force in the horizontal direction.<br />
<br />
<h3>
<b><span style="font-size: large;">Inertial Measurement Sensors</span></b></h3>
The MPU9150 was used for inertial measurement sensing for this project. The MPU9150 is:<br />
<br />
<ul>
<li>3 axis accelerometer</li>
<li>3 axis gyro</li>
<li>3 axis compass</li>
<li>...all integrated in a single IC</li>
</ul>
<br />
The benefits of having these components on a single chip, is that the error caused by misaligned axis is greatly reduced and controlled by the manufacturing process. This also makes the packaging both smaller and cheaper.<br />
<br />
<br />
The sensor data is accessed through the I2C protocol, so a microcontroller is needed to perform this (and I wrote a library both in C and C++ which I will post to my github page and provide a link).<br />
<br />
<h3>
<b>Accelerometers</b></h3>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgZ7xv1VeF_IGWnnrluRheeXVUTuvoVoOroWTzON86m-g6h0LneHkBzVl11YljOssdrzBD5Jjq2gfK7kc9_5n3yu7GXUiBqtu89DGx3W_WE8MOpZPZ_OdMWe8NfwZXarz7iNaTvv5o5Vksk/s1600/IMU_AccelDiagramPNG.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgZ7xv1VeF_IGWnnrluRheeXVUTuvoVoOroWTzON86m-g6h0LneHkBzVl11YljOssdrzBD5Jjq2gfK7kc9_5n3yu7GXUiBqtu89DGx3W_WE8MOpZPZ_OdMWe8NfwZXarz7iNaTvv5o5Vksk/s1600/IMU_AccelDiagramPNG.PNG" height="180" width="200" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both;">
Accelerometers measure linear acceleration. The diagram above shows the 3-axis accelerometer</div>
<div class="separator" style="clear: both;">
placement inside the MPU9150. When an accelerometer is stationary, it measures a net acceleration the gravitational force of 9.8m/s^2 (at least on earth). The orientation of this sensor with respect to the gravitational force (let's call it earth's z-axis) can be estimated using trigonometry:</div>
<div class="separator" style="clear: both;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhiem1LP3wBb7HlZqHzfUEwLwI7BxpnSWGA3-5RgrbGfJWdanw4QRwj4pT0qI8FOrxVZ_dN8ZyTUVhK6zSgCbPxe-SS98ylzXo3KfDOCiSRE9rA55eA7lTxTTAMq7zzrmslhYSGhS4JA0Ik/s1600/equation_angleaccelerometer.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhiem1LP3wBb7HlZqHzfUEwLwI7BxpnSWGA3-5RgrbGfJWdanw4QRwj4pT0qI8FOrxVZ_dN8ZyTUVhK6zSgCbPxe-SS98ylzXo3KfDOCiSRE9rA55eA7lTxTTAMq7zzrmslhYSGhS4JA0Ik/s1600/equation_angleaccelerometer.PNG" height="48" width="320" /></a></div>
<div class="separator" style="clear: both;">
<br /></div>
<div class="separator" style="clear: both;">
This function is accurate only when there is no external forces being applied. When external forces are applied, such as the robot moving/tipping, the orientation cannot be accurately estimated.</div>
<div class="separator" style="clear: both;">
<br /></div>
<h3>
<b>Gyros</b></h3>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjbs1HUltc3XmHpxC05-1JdTfdFYkgmprcPWQOoGY6Vr2WwyLRBUpU-NDjalBw2ulj1-KXih7HAYM0rWiQEyCH6v-djeutSqcaKbAE9QS5d1uxgjLhN5GB0nfsYojFQ7v4MG4h0kr90kXdf/s1600/IMU_GyroDiagram.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjbs1HUltc3XmHpxC05-1JdTfdFYkgmprcPWQOoGY6Vr2WwyLRBUpU-NDjalBw2ulj1-KXih7HAYM0rWiQEyCH6v-djeutSqcaKbAE9QS5d1uxgjLhN5GB0nfsYojFQ7v4MG4h0kr90kXdf/s1600/IMU_GyroDiagram.PNG" height="197" width="200" /></a></div>
<b><br /></b>
Gyros measure the the rate of angular change. The above diagram shows the 3-axis gyro placement inside the MPU9150. Gyros are not susceptible to linear forces such as vibrations or movement. They are only sensitive to angular displacements. Gyros cannot be used to directly measure physical orientation, but their readings can be integrated over time.<br />
<br />
<div style="text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgWR3kx-m0ZUOZfm_oN9VxR9i8QwEcR5g6vVUj_TSCcP84M6f7tuiF4tzkRQ7V-3qwEcK-7I1BnFSyqLcql3WeHE2DoiZzdJBQ9wRg7w3Vc5xPgoEZxVBBle_dGZ3X6BqOQESRYHwWzn3s6/s1600/equation_anglegyro.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgWR3kx-m0ZUOZfm_oN9VxR9i8QwEcR5g6vVUj_TSCcP84M6f7tuiF4tzkRQ7V-3qwEcK-7I1BnFSyqLcql3WeHE2DoiZzdJBQ9wRg7w3Vc5xPgoEZxVBBle_dGZ3X6BqOQESRYHwWzn3s6/s1600/equation_anglegyro.PNG" height="37" width="200" /></a></div>
<div style="text-align: center;">
<b>dt</b> is the control loop time.</div>
<div style="text-align: center;">
<br /></div>
Gyros suffer from a phenomenon known as drift, where they show a small rate of change even though no actual rate of change is occurring physically. When integrating, this error adds up over time, causing an inaccurate estimate of angle.<br />
<br />
Also note from the 2 pictures of the axis alignment: the x-axis gyro is perpendicular to the y-z plane of the accelerometer. So the angle using the accelerometer's x-z plane (as shown in the equation) would be the same angle using the gyro's y-axis.<br />
<br />
<h3>
<b><span style="font-size: large;">Estimating Robot Pose</span></b></h3>
<div>
<b><span style="font-size: large;"><br /></span></b></div>
<div>
In order to accurately estimate angle using gyros and accelerometers, one has to combine these sensor<br />
readings together in a way that takes advantage of their strengths in order to make up for their<br />
weaknesses. This technique is known as <b>sensor fusion</b>. There are a few different pose estimation methods that have their own strengths and weaknesses. For this project, one that is easily implemented on a low-power microcontroller, called complementary filtering is used. The equation below shows how this is implemented:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiQ2WIIc9lHcTRcr4l76gydkiafE7s07MiMRUJ4sJKdyPbIOv_WFxIKYQAVjmiRsnWv6_3ClRKioiz2Q9FE-p5cQp9Fjl25Yut7gOrlsxrJf2FzPMlQhfxDMsUm8v4eGTBPIV3y9a1fouHx/s1600/eq_sensorfusion.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiQ2WIIc9lHcTRcr4l76gydkiafE7s07MiMRUJ4sJKdyPbIOv_WFxIKYQAVjmiRsnWv6_3ClRKioiz2Q9FE-p5cQp9Fjl25Yut7gOrlsxrJf2FzPMlQhfxDMsUm8v4eGTBPIV3y9a1fouHx/s1600/eq_sensorfusion.PNG" height="27" width="400" /></a></div>
<br />
Alpha is used to weight the two angles estimates before they are combined. Since the accelerometer is more prone to adding noise to the system, a large alpha is chosen to give the gyro integration more<br />
weight in the equation, effectively applying a low-pass filter on the accelerometer. For this project, an<br />
alpha of 0.99 was used (along with a control loop time of 10 milliseconds).<br />
<br />
<h3>
<span style="font-size: large;">PID Control System</span></h3>
</div>
<div>
<span style="font-size: large;"><br /></span></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiuJRAcJeXiF5_XlcRje8bhSLkB79QexrO6caBlafZI7QLrt31fvnTvbFzQNR1vdGJGsllEyl0Io265leCh6QpogjxgyRIivRFrSWrhnZIdiLIJbItcsS7E2Sr_KB6uPABM6fId7ANt7gQs/s1600/PID_Diagram.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiuJRAcJeXiF5_XlcRje8bhSLkB79QexrO6caBlafZI7QLrt31fvnTvbFzQNR1vdGJGsllEyl0Io265leCh6QpogjxgyRIivRFrSWrhnZIdiLIJbItcsS7E2Sr_KB6uPABM6fId7ANt7gQs/s1600/PID_Diagram.png" height="213" width="320" /></a></div>
<div>
<span style="font-size: large;"><br /></span></div>
<div>
Proportional-Integral-Derivative (PID) is a control loop feedback technique using the error between a<br />
set-point and observed output of a system (often referred to as the plant). A PID controller must be tuned by varying the 3 gains associated with the control algorithm: Kp, Kd and Ki. Different choices for these constants manipulate the systems response in terms of responsetime, overshoot and oscillations.<br />
<br />
Kp: The proportional term. Depends on the present error only.<br />
Ki: Integral term. Depends on the accumulation of past errors.<br />
Kd: Derivative term. Prediction of future errors.<br />
<br />
The following is a pseudo-code example of implementing a PID algorithm in a controlled loop.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi27vwXjfdsOpnmWGmJgTdk1uIju6GVPEa8g0S2nu94hqw6GnGEOBy-frSw50LYcE_cRTB9WncgqYVAJzl-q6XSwn8D5QQgitPigeMolAVcdkmwpO47U85y64dBUB2elxSmmHnq4VyJTkoH/s1600/PID_CCODE_EX.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi27vwXjfdsOpnmWGmJgTdk1uIju6GVPEa8g0S2nu94hqw6GnGEOBy-frSw50LYcE_cRTB9WncgqYVAJzl-q6XSwn8D5QQgitPigeMolAVcdkmwpO47U85y64dBUB2elxSmmHnq4VyJTkoH/s1600/PID_CCODE_EX.png" height="68" width="200" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
The output of the PID algorithm is a sum of the gain-terms multiplied by their function on the error. The error is a simple difference from the target (in this case an angle) and the observed state (the estimated angle by the sensor fusion algorithm). It is important to keep the control loop rate consistent (so use an interrupt, or don't do extra processing).</div>
<br />
<br />
<h3>
<span style="font-size: large;">Results</span></h3>
</div>
<div>
<span style="font-size: large;"><br /></span></div>
<div>
I built my robot using parts of one of my other robots, so save on cost. I built a cheap frame out of small wood squares, a dowel and hot glue (all bought at Michaels for less than $5):</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjA4O_1W80-FKnEqnrOnHL30p_6GSB8ptOuq7XcMxH9j6XMPOysx64IxyYqqdmqBWb2ZCg44IH1lgdp2LbaZxX517FiLbqJPyYurFYILhcdXv8muXJIilb14tNsBwmU30CN5X-VVYDeHspC/s1600/robot_corner_shot_1.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjA4O_1W80-FKnEqnrOnHL30p_6GSB8ptOuq7XcMxH9j6XMPOysx64IxyYqqdmqBWb2ZCg44IH1lgdp2LbaZxX517FiLbqJPyYurFYILhcdXv8muXJIilb14tNsBwmU30CN5X-VVYDeHspC/s1600/robot_corner_shot_1.JPG" height="240" width="320" /></a></div>
<div>
<span style="font-size: large;"><br /></span></div>
<div>
Starting off with just a p controller (Kp >0, Ki = Kd = 0), the system was nowhere near useful. The robot would over-shoot and bang itself on the floor. I thought it would destroy itself before I would be able to stabilize it. Fortunately, after adding some derivative gain (Kd), the robot was able to keep itself up-right, with some oscillation. When I added some disturbance (a little push) the robot would travel horizontally until it eventually fell over. After adding some integral gain (Ki) the robot seemed to stabilize itself very well. My robot at this point would re-stabilize fast enough from a slight push, with some oscillation and small travel in the horizontal direction.My first set (and my favorite) set of gains were: Kp = 8, Ki = 0.5, Kd = 10. These gains won't mean much to you, as they are fit for my robot/system, which is dependent on my control loop, motor response, motor torque, speed resolution, robot height...and it goes on...). You would have to experiment with your system for a suitable set of gains.</div>
<div>
<br /></div>
<div>
What I can do, is share my next set of gains as a comparison to show the change in performance. I thought it would be good to rid of the oscillation, so I increased the derivative gain from 15 to 60. In order to keep my robot stable I had to increase the Kp and Ki gains slightly. This was good, in that my robot did not oscillate so much while trying to keep its balance in the non-added disturbance case. And when I did add a slight push, the robot corrected its angle almost instantly. However, the robot was forced to travel in the horizontal direction for a much larger distance to maintain this response.</div>
<div>
<br /></div>
<div>
I learned that added disturbances (like pushes) is like adding energy to the system. In order to deal with the extra energy, the robot can either oscillate back and forth a few times while minimizing travel in the horizontal direction, or it had to travel in the horizontal direction quite more just to maintain the angle I had programmed.</div>
<div>
<br /></div>
<div>
Anywho, here is a video of the robot in action:<br />
<br /></div>
<div>
<div class="separator" style="clear: both; text-align: center;">
<iframe allowfullscreen='allowfullscreen' webkitallowfullscreen='webkitallowfullscreen' mozallowfullscreen='mozallowfullscreen' width='320' height='266' src='https://www.youtube.com/embed/w2DpNIaAsqY?feature=player_embedded' frameborder='0'></iframe></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
I have a github site, so I'll post the code there. I have C, and C++ library for the MPU9150. The C library however is dependent upon the PSOC4, but you can strip what you need rather easily from it. I'll also include an application I wrote in processing (see processing.org) which the robot used to communicate to my PC so I could understand what was going on a little better:</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
Link to the code:</div>
<div class="separator" style="clear: both; text-align: left;">
<a href="https://github.com/johnnyonthespot/self-balancing-robot-psoc4/tree/master">https://github.com/johnnyonthespot/self-balancing-robot-psoc4/tree/master</a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
Thanks!</div>
<br /></div>
Brian Dwyerhttp://www.blogger.com/profile/09292143349696415766noreply@blogger.com2tag:blogger.com,1999:blog-8260024469636924539.post-36153626122429477142014-07-29T11:06:00.002-04:002014-07-29T11:16:23.692-04:00Autonomous Robot with ZynqIn an earlier post, I talked up the Xilinx Zynq (an IC with both FPGA and Microcontroller). In my Advanced Embedded System Design course, we had to build an autonomous robot that can navigate around with some form of intelligence and seek out certain objects to 'destroy'. Now the term 'destroy' was really left up to the students to define; for our robot a laser pointer was used to mark enemies. Identifying 'enemies' was a challenge, so we incorporated a camera so the robot could see and track on its own. Meet our robot (below):<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhEzNrMhm8kJbKQHfFs4Vq7DVeXZkqNWp_3swjCUiZH2-B8_VZ0glf0vLwKsTLG-SygBK2z7dF1QVFYthhwMf0oS_nBbL2u-X42nvwCD0bR9XP8dedX01XaGfbWal16wi8ZxUtt58pLx19i/s1600/robot.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhEzNrMhm8kJbKQHfFs4Vq7DVeXZkqNWp_3swjCUiZH2-B8_VZ0glf0vLwKsTLG-SygBK2z7dF1QVFYthhwMf0oS_nBbL2u-X42nvwCD0bR9XP8dedX01XaGfbWal16wi8ZxUtt58pLx19i/s1600/robot.png" height="456" width="640" /></a></div>
<br />
Equipped with:<br />
<br />
<ul>
<li> <a href="http://www.digilentinc.com/Products/Detail.cfm?NavPath=2,400,1198&Prod=ZYBO">Zybo</a></li>
<li>OV7670 Camera</li>
<li><a href="http://www.arduino.cc/">Arduino</a></li>
<li><a href="http://www.robotshop.com/en/sabertooth-dual-regenerative-motor-driver.html?utm_source=google&utm_medium=base&utm_campaign=GoogleUSA">Sabertooth</a> Motor Controller</li>
<li>IR Proximity Sensors</li>
<li>LiPo Batteries (12V + 7V)</li>
<li>Pan/Tilt Servo motors</li>
<li>Laser</li>
</ul>
Custom logic was designed onto the FPGA portion of the Zynq, in order to maneuver the robot, control the pan/tilt bracket, capture frame data from the camera, and lastly - 'fire the laser'. The ARM portion of the Zynq was used as the algorithm prototyping environment (in C) to make use of the custom FPGA interfaces. The Arduino was used to configure the OV7670 over the I2C-like communications. The following diagram shows how all components were interfaced.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiN_TmxUa4V802TThF-u9jgxMOwVsM0tsJ_WvcdDNCV3buBs5CF-oEW8fn6vvpzMiOL8KsRpJCN8eIx5CAZElWOONd5jWFsr-KKJOVGbVlRQO0WTPQ4J_NCImdxted3E3IRdVtM6wCXyqj4/s1600/diagram.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiN_TmxUa4V802TThF-u9jgxMOwVsM0tsJ_WvcdDNCV3buBs5CF-oEW8fn6vvpzMiOL8KsRpJCN8eIx5CAZElWOONd5jWFsr-KKJOVGbVlRQO0WTPQ4J_NCImdxted3E3IRdVtM6wCXyqj4/s1600/diagram.png" height="324" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
The reason the OV7670 was chosen was its parallel data interface and low cost ($20), allowing us to obtain an image capture rate of 30fps. However, you get what you pay for in terms of ease of interfacing. I had to design custom logic in VHDL to perform frame captures and store them in block-ram on the FPGA. It's hard to debug what you can't see. None the less, after days of toying around, the Zybo was finally able to see. In order to view what the Zybo saw, I wrote a quick program to transfer images from the Zynq to my laptop over a serial connection (using <a href="http://processing.org/">Processing</a>). Below is the process flow for debugging the images.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhLrFvoRs-Qfk_u6wjZ0wGTlX_IBSJ16Cs6glwff9gWpJOPnROSp9uGfmMrI9krclURRzRrLeYrjXYlQfnNpC53igQQgL4yN4pRPUXhOFH75k8kmHWmvMkw-5jV0nPBcTXTstul_I_XWAeP/s1600/imageprocessflow.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhLrFvoRs-Qfk_u6wjZ0wGTlX_IBSJ16Cs6glwff9gWpJOPnROSp9uGfmMrI9krclURRzRrLeYrjXYlQfnNpC53igQQgL4yN4pRPUXhOFH75k8kmHWmvMkw-5jV0nPBcTXTstul_I_XWAeP/s1600/imageprocessflow.png" height="315" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
The custom FPGA logic which interfaced to the OV7670 was designed to either stream frames into the block-ram, or take a single snapshot and leave it in the block-ram. The FPGA had to interface/synchronize to the vsync, href, pixel-clock, and 8-bit data of the OV7670. It also needed to interface with the ARM processor and to FPGA-based block-RAM. Below is the block model of the custom camera control as shown in <a href="http://www.xilinx.com/products/design-tools/vivado/">Vivado</a>.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiqvUu2dR0m0KEr1i_XmEUBdI1_9G36m2LrHxfuUmrI4TXj7Lstlcg2A5gb6NuiHy-Juo61nCbtk09a_Cb306BfoT_tJ0WDIEjbMGvvIBnR9O2tctOnipkLIS0gKzXYyDCULXpLLs8YkxQk/s1600/vivado_om7670.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiqvUu2dR0m0KEr1i_XmEUBdI1_9G36m2LrHxfuUmrI4TXj7Lstlcg2A5gb6NuiHy-Juo61nCbtk09a_Cb306BfoT_tJ0WDIEjbMGvvIBnR9O2tctOnipkLIS0gKzXYyDCULXpLLs8YkxQk/s1600/vivado_om7670.png" height="264" width="320" /></a></div>
We couldn't have done it without camera register settings provided by this source: <a href="http://hamsterworks.co.nz/mediawiki/index.php/OV7670_camera">Hamsterworks - OV7670</a><br />
<br />
Otherwise the images didn't turn out so well:<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgBVXjekmFRx0HOzeenCvZNli9unbxoWnIeUx0VGS6y5Qe0p7SB3r4tYj1VXrsQB7MdDl8jxLO8x0efJaPU7aFIhS6vpoZj-ha6XIELPipVjPnViHGh7zonVs4dE9ACr7fEhrLfuj8tHLXR/s1600/zybo_funny.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgBVXjekmFRx0HOzeenCvZNli9unbxoWnIeUx0VGS6y5Qe0p7SB3r4tYj1VXrsQB7MdDl8jxLO8x0efJaPU7aFIhS6vpoZj-ha6XIELPipVjPnViHGh7zonVs4dE9ACr7fEhrLfuj8tHLXR/s1600/zybo_funny.png" height="271" width="320" /></a></div>
<br />
I'll post some videos of the robot in action soon...Brian Dwyerhttp://www.blogger.com/profile/09292143349696415766noreply@blogger.com0tag:blogger.com,1999:blog-8260024469636924539.post-10588378058268570522014-04-29T14:54:00.001-04:002015-12-18T02:27:58.798-05:00Xilinx Zynq7000 and the ZyboIf you are into getting your hands on the latest embedded technology, the Zynq7000 is a great platform to get familiar with. Xilinx, a leader in FPGA design (in which I have no affiliation with), partnered up with ARM to create the very first microprocessor with on-chip FPGA (or is it an FPGA with an on-chip processor?).<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh1-YBgZs_QIjQjwsDEfMzcT4O7uD4xEtfQt9X0Im7vIZR0CM-mWV2J6CbAXcH8g0UH3NP7Dg5ETimcoCIQjjyMsjexPXhIYNR6HfLUOc_Gv1E_jmlJof2Sd9dwVi-FDFfB03oZOwt9lOkC/s1600/zynq.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh1-YBgZs_QIjQjwsDEfMzcT4O7uD4xEtfQt9X0Im7vIZR0CM-mWV2J6CbAXcH8g0UH3NP7Dg5ETimcoCIQjjyMsjexPXhIYNR6HfLUOc_Gv1E_jmlJof2Sd9dwVi-FDFfB03oZOwt9lOkC/s400/zynq.jpg" width="355" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://www.xilinx.com/products/silicon-devices/soc/zynq-7000/" target="_blank">Xilinx Zynq7000</a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
This technology combines the power of using an FPGA to perform high-speed paralleled tasks with a dual-core ARM processor with standard Micorcontroller peripherals (CAN, I2C, SPI, UART etc.). I was lucky enough to be introduced to this device in my Advanced Embedded System Design course at Oakland University, and will continue working with it as long as I can continue to make sense out of partitioning my embedded design projects into both 'some hardware' and 'some software'. I purchased the Zybo development board made by Digilient, however there are others out there (Zedboard, microZed by AVNet)<br />
<br />
To paint a better picture of how this is so useful, imagine using a microcontroller to both interface to a camera and perform image processing in order to make a decision (maybe you are trying to track an object). In terms of cameras you are able to interface to, you are left with those with slower serial interfaces (SPI, UART). If you really wanted to interface a microcontroller to a camera with a faster parallel interface, you would have to spend a little more money to get a fast enough controller. But this doesn't leave your controller much time to do anything else other than image-processing and frame-grabbing! On the flip side, if you were to use an FPGA it becomes harder to actually implement your decision making and takes longer to develop and is harder to debug.<br />
<br />
Having an FPGA and Microcontroller on the same IC solves this exact problem. Now, you can develop your frame-grabbing for your higher-speed camera and even do some image processing on the FPGA. In this way, the FPGA can be treated as a custom co-processor for your application running on the Processor. And the FPGA to Processor interface is seen by your application is either 'just another memory mapped peripheral' or even an 'external interrupt'.<br />
<br />
The example I used comes directly from my project, where I did use the FPGA to communicate with a faster camera (well faster than UART or SPI based) and allow the processor to dictate what kind of operations to perform on the image, or even stream the image to the processor's RAM for streaming to a remote PC via UART. This was for an Autonomous Robot project, which I will add posts on in the near future.<br />
<br />
I recommend checking this technology out. It is great to see innovations such as this, because they have the power to take industries into new directions. If you are interested in getting yourself one, I suggest either the Zedboard or the Zybo.<br />
<br />
<a href="http://www.digilentinc.com/Products/Detail.cfm?NavPath=2,719,1197&Prod=ZYBO" target="_blank">Link to Digilent's Zybo</a><br />
<br />Brian Dwyerhttp://www.blogger.com/profile/09292143349696415766noreply@blogger.com0tag:blogger.com,1999:blog-8260024469636924539.post-4566235099804136892012-04-20T01:09:00.001-04:002015-01-02T11:28:14.409-05:00Audio LocalizationJust a little introduction on my interest in this topic:<br />
I've always been intrigued by the way our mind is configured to interpret sound signals. To those of you with two working ears: Ever notice when you hear a noise, you know which direction it came from?... I mean you just 'know'; you don't have to sit down, grab a pencil and notepad and plot waveforms to triangulate the angle in which the sound likely originated. These calculations are done in the background of our mind. That's right, you and I (and even our pet cats) are pre-programmed to utilize these functions without having to 'think' about it. This way we can use our main processor-time for more important tasks.<br />
<br />
I wanted to experiment with methods that the brain uses for indicating the direction of sound. A little background on the two methods:<br />
<ul>
<li>Interaural Level Difference: used to describe difference in amplitude between two or more sensors</li>
<li>Interaural TIme Difference: used to describe difference in arrival time of two signals</li>
</ul>
A few links on the topic:<br />
<a href="http://www.ise.ncsu.edu/kay/msf/sound.htm" target="_blank">http://www.ise.ncsu.edu/kay/msf/sound.htm</a><br />
<a href="http://en.wikipedia.org/wiki/Sound_localization" target="_blank">http://en.wikipedia.org/wiki/Sound_localization</a><br />
<ul>
</ul>
For simplicity, I focused on the 1st method and implemented an 'object tracking' approach.<br />
<br />
Components used:<br />
<ul>
<li>Arduino Uno</li>
<li>2 Phidget Sound Sensors</li>
<li>Continuous Servo Motor</li>
</ul>
To continue to maintain simplicity, I chose the Phidget Sound Sensors because they outputs a 0-5 Volt signal representing measured volume (opposed to a raw signal from a microphone). This also allows for a slower processor (such as the Atmega328) to be quick enough for the task. Below is a pic of the system (made for a class project).<br />
<br />
Here is a functional diagram of the system I drew up:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi1RTLTgLTZGGu0dW_tH4P8sxv_HwlZMl7ojBeCrnRey_UIojmBNRWyy9Yz078kAhS1ntog1p5As8ZTJmSkLcG0xC4hL2W3SeXatUfrDb7nfFOWmyFQlk8JhlbVQkK0y-iflnsIrvomvYzs/s1600/AudioDiagram.bmp" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi1RTLTgLTZGGu0dW_tH4P8sxv_HwlZMl7ojBeCrnRey_UIojmBNRWyy9Yz078kAhS1ntog1p5As8ZTJmSkLcG0xC4hL2W3SeXatUfrDb7nfFOWmyFQlk8JhlbVQkK0y-iflnsIrvomvYzs/s320/AudioDiagram.bmp" height="223" width="320" /></a></div>
<div style="text-align: left;">
A high-level schematic<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEikiofZCEYP2UF7jsy1gSZHibEncOrAN3gOrSXW8pZ85Vv7nV9dIHxlVkrylgQJon178rKboC9rYVh7SOB2LBgQ-s9pObCpa8dSr8KYggzIp4Ze975w8tDkGxvQXKLwp6fg2fX0DKgDyKYJ/s1600/schematic.bmp" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEikiofZCEYP2UF7jsy1gSZHibEncOrAN3gOrSXW8pZ85Vv7nV9dIHxlVkrylgQJon178rKboC9rYVh7SOB2LBgQ-s9pObCpa8dSr8KYggzIp4Ze975w8tDkGxvQXKLwp6fg2fX0DKgDyKYJ/s320/schematic.bmp" height="172" width="320" /></a></div>
<br />
A picture of the project</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiTCGfD3u3fO3CY04oHWLZrS29WY8kI_bechJ7YQ2sMSUURdQSqT1jxPQnBEAKMkTIg-9qTBK1G7JIBsdvxDAHkLXRvthVM2IAmabVVm7KDUPBOn-tGidisdJT1A_6EuB7by2i4OGigHNyp/s1600/IMAG0586.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiTCGfD3u3fO3CY04oHWLZrS29WY8kI_bechJ7YQ2sMSUURdQSqT1jxPQnBEAKMkTIg-9qTBK1G7JIBsdvxDAHkLXRvthVM2IAmabVVm7KDUPBOn-tGidisdJT1A_6EuB7by2i4OGigHNyp/s320/IMAG0586.jpg" height="191" width="320" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
And at last, a video! (May be loud). We used the "Air-Horn" Phone-App</div>
<div class="separator" style="clear: both; text-align: center;">
<object class="BLOGGER-youtube-video" classid="clsid:D27CDB6E-AE6D-11cf-96B8-444553540000" codebase="http://download.macromedia.com/pub/shockwave/cabs/flash/swflash.cab#version=6,0,40,0" data-thumbnail-src="http://i.ytimg.com/vi/GbMaHHulh_s/0.jpg" height="266" width="320"><param name="movie" value="http://www.youtube.com/v/GbMaHHulh_s?version=3&f=user_uploads&c=google-webdrive-0&app=youtube_gdata" />
<param name="bgcolor" value="#FFFFFF" />
<embed width="320" height="266" src="http://www.youtube.com/v/GbMaHHulh_s?version=3&f=user_uploads&c=google-webdrive-0&app=youtube_gdata" type="application/x-shockwave-flash"></embed></object></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://youtu.be/GbMaHHulh_s" target="_blank">http://youtu.be/GbMaHHulh_s</a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
Video with improved code<br />
<div class="separator" style="clear: both; text-align: center;">
<object class="BLOGGER-youtube-video" classid="clsid:D27CDB6E-AE6D-11cf-96B8-444553540000" codebase="http://download.macromedia.com/pub/shockwave/cabs/flash/swflash.cab#version=6,0,40,0" data-thumbnail-src="http://i.ytimg.com/vi/_GYjfpB4mR0/0.jpg" height="266" width="320"><param name="movie" value="http://www.youtube.com/v/_GYjfpB4mR0?version=3&f=user_uploads&c=google-webdrive-0&app=youtube_gdata" />
<param name="bgcolor" value="#FFFFFF" />
<embed width="320" height="266" src="http://www.youtube.com/v/_GYjfpB4mR0?version=3&f=user_uploads&c=google-webdrive-0&app=youtube_gdata" type="application/x-shockwave-flash"></embed></object></div>
<div class="separator" style="clear: both; text-align: left;">
</div>
Brian Dwyerhttp://www.blogger.com/profile/09292143349696415766noreply@blogger.com2tag:blogger.com,1999:blog-8260024469636924539.post-83414146237405377762011-05-03T21:47:00.011-04:002015-01-02T11:30:50.407-05:00Return of the RC car!!!I've been very busy since the last post, but I finaly got back into the groove for the RC car. I wanted this car to be completely modifiable so the inside of the electronics control box = breadboards + velcro (which works pretty darn good). Here is a photo of the wiring. <br />
<br />
Included in this box:<br />
1. Boarduino (Atmega 328) controller<br />
2. 18v15 Pololu Motor Driver (rear motor drive)<br />
3. Dual AxisCompass Module - HMC6352<br />
4. Xbee Pro 60mW U.FL module: for wireless comunication <br />
<br />
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhtN_JSJ89-vDOvWFD0C36AC44cNpNMbQXclLbLqjvywWbLTUenxNQuCeQKqktn4ADQao8n_2BwxhV3q7Cx-C55kcDovMF-YWRyloJliQo3PDdMWNVJCknEU8MoZPgEO6NIZMm8GpxONsDd/s1600/CarWiring1.jpg"><img alt="" border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhtN_JSJ89-vDOvWFD0C36AC44cNpNMbQXclLbLqjvywWbLTUenxNQuCeQKqktn4ADQao8n_2BwxhV3q7Cx-C55kcDovMF-YWRyloJliQo3PDdMWNVJCknEU8MoZPgEO6NIZMm8GpxONsDd/s400/CarWiring1.jpg" id="BLOGGER_PHOTO_ID_5602679057164135858" style="cursor: hand; display: block; height: 239px; margin: 0px auto 10px; text-align: center; width: 400px;" /></a> <br />
<br />
I had to replace the stock steering motor/potentiometer setup with a servo motor to put ease on the programming (I'm now able to eliminate the steering control loop code). I also broke the ICSP header outside the box for programming (I had to wire up a switch to disconnect both the external power and the Xbee's transmit line from the microcontroller in order to externally program). <br />
<br />
I'm working on the code for wireless control/feedback. Eventually I intend on adding a camera, ultrasonic rangefinder and GPS module. <br />
<br />
Anyway, here is a short video on my car's first run with both functional steering/driving. (given the small area, I had it just do circles in my kitchen).<br />
<iframe allowfullscreen='allowfullscreen' webkitallowfullscreen='webkitallowfullscreen' mozallowfullscreen='mozallowfullscreen' width='320' height='266' src='https://www.blogger.com/video.g?token=AD6v5dzyI1YJ8a_nirZoA37sXN0hddhS4wU3CnyiBSIE-e1n-ebxZf7THYorzqRsCzD2vwijiLKlRSsDBarDeDet_g' class='b-hbp-video b-uploaded' frameborder='0'></iframe>Brian Dwyerhttp://www.blogger.com/profile/09292143349696415766noreply@blogger.com0tag:blogger.com,1999:blog-8260024469636924539.post-54056097602985800102010-08-04T21:20:00.005-04:002015-01-02T11:30:19.558-05:00RC CarI'm currently putting together a new project where I'm attempting to automate the control of an old RC Car. I've ripped out the original guts except for the the drive motor and steer motor. After numerous attempts at a DIY motor controller for the drive motor (and popping through transistors like popcorn), I've come to the conclusion that this motor needed a driver who could handle the spike currents without melting under pressure. Here is a list of all the parts that will be going into this project, and later I will provide updates on my progress.<br />
<br />
Parts:<br />
<br />
1. Original RC Chasis (including suspension and wheels/tires).<br />
2. Original RC 7.2V Battery Packs (2)<br />
3. Original rear-wheel drive motor (specs unknown except that it draws 3.5A current running, and 15A stall).<br />
4. Orignal steering motor + original feedback potentiometer (specs also unkown)<br />
5. Boarduino (Atmega 328) controller<br />
6. 18v15 Pololu Motor Driver (to properly drive the rear motor)<br />
7. Dual AxisCompass Module - HMC6352<br />
8. Xbee Pro 60mW U.FL modules + antenna (2): for wireless comunication<br />
9. A few 2n2907/2n2222 BJTs for driving the steering motor<br />
<br />
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj8XzNzTnh34w2WTm-HQgYZ2OkxMOv0hQ5XfaEp8ehxBX-3aai6gKBUKEU_TCu8_HJ1mg1TUMNUckC1temlYuMzBcpbjeASBW0YALovap7rzfilHGKWpCfHQR2p5UUC6Umy3Crj06L_O8Nn/s1600/1.JPG"><img alt="" border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj8XzNzTnh34w2WTm-HQgYZ2OkxMOv0hQ5XfaEp8ehxBX-3aai6gKBUKEU_TCu8_HJ1mg1TUMNUckC1temlYuMzBcpbjeASBW0YALovap7rzfilHGKWpCfHQR2p5UUC6Umy3Crj06L_O8Nn/s320/1.JPG" id="BLOGGER_PHOTO_ID_5501732287046530386" style="cursor: hand; cursor: pointer; display: block; height: 214px; margin: 0px auto 10px; text-align: center; width: 320px;" /></a><br />
<br />
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjpJUYdVyCJs-QDNmbLjd8dbaPgOtVMMu7XtZKrqqES7Trv3lpCt7nBlT2kJmOPbM9-PAw3GpKwHO8V0bm81RhHoPmrh6l1PX7XqlSTK9bEDClbah6h8MYxIyYSd49KIiBmTsgXjpwAlgDk/s1600/2.JPG"><img alt="" border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjpJUYdVyCJs-QDNmbLjd8dbaPgOtVMMu7XtZKrqqES7Trv3lpCt7nBlT2kJmOPbM9-PAw3GpKwHO8V0bm81RhHoPmrh6l1PX7XqlSTK9bEDClbah6h8MYxIyYSd49KIiBmTsgXjpwAlgDk/s400/2.JPG" id="BLOGGER_PHOTO_ID_5501733461647661170" style="cursor: hand; cursor: pointer; display: block; height: 267px; margin: 0px auto 10px; text-align: center; width: 400px;" /></a>Brian Dwyerhttp://www.blogger.com/profile/09292143349696415766noreply@blogger.com0tag:blogger.com,1999:blog-8260024469636924539.post-90624682955937576292010-04-25T01:23:00.011-04:002015-01-02T11:28:49.920-05:00Light Follower - 2 Axis<img alt="" border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhsT91SoyhyphenhyphenDDUJH7sXqEXpD94pGK4K8lA_wFtdp1ZNcD9FY212YbS5uWs28Qfx5Zvl81LpQBrRLF4naEFw3lVGlnmE0UY02Bg8WnnpWgPVlo209U6Zb1ntY0-NnTBQZbIx8Ifsda3-Vo19/s320/setup1.JPG" id="BLOGGER_PHOTO_ID_5463944400728215026" style="cursor: hand; display: block; height: 214px; margin: 0px auto 10px; text-align: center; width: 320px;" /><br />
<div align="left">
<div>
Here it is - My rendition of a 2-axis light follower. I went ahead and clipped the IR Cam from the wiimote to cut down on weight. I put the camera along with the required oscillator in a project box (from radio shack). I also included an LED on the box; it lights up when light is seen by the camera. A serial port is used to connect the box to the microcontroller.</div>
<div>
</div>
<div>
Here are some Pictures:</div>
<br />
<div>
</div>
<img alt="" border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjGqJelmAzX3j8q2iZyYnFd-1qdgRMUGfwJ54s3sLvAEcqyt2ggmTgMHnc5OywLwiSGPvNjZglPBwRfU6jWnkYWor_81VBmE5-WCZkrf6veHd__nO7wvHQ3ofqGxPgB0mMnM6V4W_hIHf2i/s320/Inside+Camera.JPG" id="BLOGGER_PHOTO_ID_5463945322145565186" style="cursor: hand; display: block; height: 214px; margin: 0px auto 10px; text-align: center; width: 320px;" /><br />
<div align="left">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgSQ9g7b0483mQSsCduR2fqyq4jLPUi2eQ7-VD2mpBMe1ihmoAU1rpKFE6_nI6o-k6-h73kpX-tOmQBXJy2mSg3c-nUMicY8lB9NhVLizTFatUtqkg3BZ9Pac5I5gOjEup9g2kx2ThD8vt0/s1600/Parts.JPG"><img alt="" border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgSQ9g7b0483mQSsCduR2fqyq4jLPUi2eQ7-VD2mpBMe1ihmoAU1rpKFE6_nI6o-k6-h73kpX-tOmQBXJy2mSg3c-nUMicY8lB9NhVLizTFatUtqkg3BZ9Pac5I5gOjEup9g2kx2ThD8vt0/s320/Parts.JPG" id="BLOGGER_PHOTO_ID_5463943741801855666" style="cursor: hand; display: block; height: 214px; margin: 0px auto 10px; text-align: center; width: 320px;" /></a></div>
<div align="left">
Here is a video:</div>
<div align="center">
<iframe allowfullscreen='allowfullscreen' webkitallowfullscreen='webkitallowfullscreen' mozallowfullscreen='mozallowfullscreen' width='320' height='266' src='https://www.blogger.com/video.g?token=AD6v5dyA8FszCKBF7rAr4cWqP32zPLQkB2C3y79zejCJ4FhhD1chvHV3rQoe9VNYREgcldPD2QmPlLuAC47Pah1eSA' class='b-hbp-video b-uploaded' frameborder='0'></iframe></div>
<div align="center">
<a href="http://www.youtube.com/watch?v=y41o7KieRgw" target="_new">http://www.youtube.com/watch?v=y41o7KieRgw</a></div>
<div align="left">
</div>
<div align="left">
Here is a link to the code:</div>
<div align="left">
<a href="http://www.wiimoteproject.com/wiimote-accelerometer-and-motions-detecting-projects/light-follower-2-axis/">http://www.wiimoteproject.com/wiimote-accelerometer-and-motions-detecting-projects/light-follower-2-axis/</a></div>
</div>
Brian Dwyerhttp://www.blogger.com/profile/09292143349696415766noreply@blogger.com6tag:blogger.com,1999:blog-8260024469636924539.post-31988863935717394072010-03-17T02:26:00.006-04:002010-03-17T02:37:07.171-04:00Nintendo nunchucks as Orientation SensorsMy senior design project was to make a wirelessly controlled robotic arm, that mimics human arm movements. The closest we got was the movement of a shoulder joint and an elbow joint at a very high accuracy and low time delay.<br /><br />I made a sensor system out of 2 Wii Nunchucks, an Arduino and some external circuitry to switch between nunchuck sensor reading. At a deadline of one of our presentations, a plastic gear from our arm had chipped some of its teeth so we weren't able to give motion demos. I whipped up a program using Processing (processing.org) to communicate with the Arduino and move a 3D simulation of an arm based on the sensor outputs.<br /><br />I thought I'd share a few screenshots of the program. I'll have both the Arduino code, and processing code up soon.<br /><br /><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh25iL2T7TdmGSXCY95CcJ07yFQSVdtz8dEh6o9UbJQypg0RkOZ3SkvlvZOJ8kSwcoUKDf2Xd1eXOdP0gl0hcaniDxjiFbtVVtRMRiKqDDrz8dvkn_ArGkm56oNspVjXlGr3z5QHe1kNHel/s1600-h/wrist.jpg"><img id="BLOGGER_PHOTO_ID_5449487608098059090" style="DISPLAY: block; MARGIN: 0px auto 10px; WIDTH: 400px; CURSOR: hand; HEIGHT: 248px; TEXT-ALIGN: center" alt="" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh25iL2T7TdmGSXCY95CcJ07yFQSVdtz8dEh6o9UbJQypg0RkOZ3SkvlvZOJ8kSwcoUKDf2Xd1eXOdP0gl0hcaniDxjiFbtVVtRMRiKqDDrz8dvkn_ArGkm56oNspVjXlGr3z5QHe1kNHel/s400/wrist.jpg" border="0" /></a><br /><div><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg53VmhJ1gDMZDWr2tX2p9CsJymZTwTDZ_pzI-0agmiBBNJMWVqH5_j6Ioyo1FVB8C1yE9BAj8-vBhIRqhVkWQPUxzbwRNwWLh7MLJ1x5BMltcstSZaDcW7pskFS9ohy9MGcR0bokFSYTNx/s1600-h/uarmfarm.jpg"><img id="BLOGGER_PHOTO_ID_5449487542046225938" style="DISPLAY: block; MARGIN: 0px auto 10px; WIDTH: 400px; CURSOR: hand; HEIGHT: 332px; TEXT-ALIGN: center" alt="" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg53VmhJ1gDMZDWr2tX2p9CsJymZTwTDZ_pzI-0agmiBBNJMWVqH5_j6Ioyo1FVB8C1yE9BAj8-vBhIRqhVkWQPUxzbwRNwWLh7MLJ1x5BMltcstSZaDcW7pskFS9ohy9MGcR0bokFSYTNx/s400/uarmfarm.jpg" border="0" /></a><br /><br /><div> </div><div><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEifVALKsZnhNyIYfipTw0Zpg2sUbjURojXZu_tXWlK_2IFWYNCDgzivTBa8N4OwsES94-0Imtrfsa3X7Cg8LkuBxfQwmVAYnvddt6-PdlRSZDqLbdTiMGjwlcqchTbgiUTSim5aTI4rbOGX/s1600-h/elbow.jpg"><img id="BLOGGER_PHOTO_ID_5449487454794654466" style="DISPLAY: block; MARGIN: 0px auto 10px; WIDTH: 400px; CURSOR: hand; HEIGHT: 338px; TEXT-ALIGN: center" alt="" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEifVALKsZnhNyIYfipTw0Zpg2sUbjURojXZu_tXWlK_2IFWYNCDgzivTBa8N4OwsES94-0Imtrfsa3X7Cg8LkuBxfQwmVAYnvddt6-PdlRSZDqLbdTiMGjwlcqchTbgiUTSim5aTI4rbOGX/s400/elbow.jpg" border="0" /></a></div></div><br /><p> </p><p>Processing is a great program for doing graphical manipulations. It can also compile code to execuatable files.</p>Brian Dwyerhttp://www.blogger.com/profile/09292143349696415766noreply@blogger.com0tag:blogger.com,1999:blog-8260024469636924539.post-22381652180383187792010-03-17T00:03:00.001-04:002015-01-02T11:29:47.920-05:00Wiimote light follower with servoEverybody is familiar with the infamous Wiimote. When I look at it, I think about all the useful sensors/gadgets that this little 40$ package (new) comes with. Recently I've been playing with the IR Camera (It's really just a light sensing camera with an IR Filter).This particular camera is a standalone module that outputs coordinates of the 4 brightest "images", all via I2C communication.<br />
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg52ui9LSxxwI9Mfeh0WLHd5vRpsDO6qENO6mWkmaDZSO0oQvSLSMup7XYB27_aOU5pzIu6DOtEoEDxNOF077HprG4txkWjcjXtnerkh84KgoddA-ebPvlEz4QzlfoETu-9zskFm6-ru_ek/s1600-h/top.JPG"><img alt="" border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg52ui9LSxxwI9Mfeh0WLHd5vRpsDO6qENO6mWkmaDZSO0oQvSLSMup7XYB27_aOU5pzIu6DOtEoEDxNOF077HprG4txkWjcjXtnerkh84KgoddA-ebPvlEz4QzlfoETu-9zskFm6-ru_ek/s400/top.JPG" id="BLOGGER_PHOTO_ID_5449455394347295842" style="cursor: hand; display: block; height: 400px; margin: 0px auto 10px; text-align: center; width: 267px;" /></a><br />
I've only seen hacks with the Wiimote cam where the camera is desoldered/removed from the Wiimote. However, at 40$ a pop that seemed like a waste of a perfectly good Wiimote. Instead of removing the cam, I only made 1 small modification, which was drilling a very tiny hole near the camera and soldering a connection to it's "Clock" pin (which needs to be a 24MHz sine wave to replace the internal oscillator). Once you have this done, all you need to do is plug a cord into the Wiimote peripheral port to use anything on its I2C bus.<br />
Moving on, I attached the Wiimote on a homemade stand that was fixed onto a continuous-rotation servo motor (servo without feedback). Add a little duct tape, and that servo isn't moving for at least an hour.<br />
<br />
<div>
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhSUvBTOIs0dcna68DnlAdzKyKJMgaMRD1tdrgDAfd58mS2v8Hz1RGBvRZxSD5P93hCW2EXvmT7yJTaN-jLFdpWRNGnniD-fRjti72GZZ1VbCx_Y3aMH09xA11-D445fWHzGhYIIG3j2XUO/s1600-h/side+view.JPG"><img alt="" border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhSUvBTOIs0dcna68DnlAdzKyKJMgaMRD1tdrgDAfd58mS2v8Hz1RGBvRZxSD5P93hCW2EXvmT7yJTaN-jLFdpWRNGnniD-fRjti72GZZ1VbCx_Y3aMH09xA11-D445fWHzGhYIIG3j2XUO/s400/side+view.JPG" id="BLOGGER_PHOTO_ID_5449454973304513730" style="cursor: hand; display: block; height: 267px; margin: 0px auto 10px; text-align: center; width: 400px;" /></a><br />
Using an IR Camera library already created (thanks to Hobley – <a href="http://www.stephenhobley.com/">http://www.stephenhobley.com/</a>), I used an Arduino to receive points from the camera and follow the 1st object (light source) that's noticed. </div>
<div>
<br />
The code can be found here:</div>
<div>
<a href="http://www.wiimoteproject.com/wiimote-accelerometer-and-motions-detecting-projects/wiimote-light-follower-with-servo">http://www.wiimoteproject.com/wiimote-accelerometer-and-motions-detecting-projects/wiimote-light-follower-with-servo</a></div>
<div>
</div>
<br />
<iframe allowfullscreen='allowfullscreen' webkitallowfullscreen='webkitallowfullscreen' mozallowfullscreen='mozallowfullscreen' width='320' height='266' src='https://www.blogger.com/video.g?token=AD6v5dxa2vIgRjSFMFE3jU6012hjPaHe9lHf6umyOH80ub1ID2-VGA5JzzS3PD2Pe9vwW8APCMKSfT3PzonBLGJX5g' class='b-hbp-video b-uploaded' frameborder='0'></iframe><br />
<a href="http://www.youtube.com/watch?v=Nj7UqjP-z6U">http://www.youtube.com/watch?v=Nj7UqjP-z6U</a>Brian Dwyerhttp://www.blogger.com/profile/09292143349696415766noreply@blogger.com8tag:blogger.com,1999:blog-8260024469636924539.post-33379080321537097442010-03-15T23:07:00.000-04:002015-01-02T11:29:32.606-05:00Two Wii Nunchucks with one arduinoIn the midst of a senior design project, it was decided that we wanted to use 2 Wii Nunchucks as accelerometers to measure orientation of a human arm ( 1 for the upper arm and 1 for the forearm). In understanding I2C communication, there is no way to use 2 Nunchucks on the same I2C bus without some sort of external circuitry. (All nunchucks have the same slave address, leaving nothing to distinguish between the two when attempting to receive data).<br />
<br />
I drew up a simple and cheap solution to interface two (or more) Wii Nunchucks on the same I2C bus. This is useful for projects that require multiple accelerometers at a cheap price.<br />
<br />
Here is all you will need:<br />
<br />
2 npn switching transistors (I used 2N3904)<br />
2 current limiting resistors (I used 200 Ohm)<br />
<br />
<br />
<br />
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj0eNsmZSh2LiwKG9H0nj1JYenH7azymZItkgtuNUkdJAqHg0snIgXRalOOkkw5z2Ws0AWPXbOOqfDeE_gPlLuNpPUJxXw6uPnP_PFI5AaK-77Y5cG8L64gRbtRnHg9V442QHqm1LDVyNv6/s1600-h/diagram2nunchucks.jpg"><img alt="" border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj0eNsmZSh2LiwKG9H0nj1JYenH7azymZItkgtuNUkdJAqHg0snIgXRalOOkkw5z2Ws0AWPXbOOqfDeE_gPlLuNpPUJxXw6uPnP_PFI5AaK-77Y5cG8L64gRbtRnHg9V442QHqm1LDVyNv6/s400/diagram2nunchucks.jpg" id="BLOGGER_PHOTO_ID_5449068941803477730" style="cursor: hand; display: block; height: 319px; margin: 0px auto 10px; text-align: center; width: 391px;" /></a><br />
Just connect all nunchuck Power (PWR), Clock (SCL), and Ground (GND) wires to the same corresponding spots on your microcontroller. The microcontroller's SDL can be connected to the outputs of both transistors. <br />
<br />Programming notes: <br />
<br />In order to perform a read, all you have to do is set the pin of the corresponding transistor to HIGH (5v in our case), write/read to the I2C bus, then set that pin to LOW (0v)to disconnect that nunchuck from the bus. Also, during start up you must initialize each nunchuck individually in order to operate both nunchucks correctly.<br />
Good luck,<br />
B Dwyer (aka: johnnyonthespot)<br />
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjEYOhpcu-vLzDczmn-bHofePZjOExf1Ch3oP_ylduFbAqcet-enYBHynbZKnMN_GHDMdzYLKfpn3n72-JIoFNgS9lL7OjkSgRqkHZkw5Bggc8vjWGEhoPF5H9rFfh8C6p8MJrhY6OTRmxO/s1600-h/diagram2nunchucks.jpg"></a>Brian Dwyerhttp://www.blogger.com/profile/09292143349696415766noreply@blogger.com8