{"id":7612,"date":"2011-06-08T10:22:18","date_gmt":"2011-06-08T08:22:18","guid":{"rendered":"https:\/\/www.adimec.com\/?p=7612"},"modified":"2018-07-27T17:14:04","modified_gmt":"2018-07-27T15:14:04","slug":"the-pros-and-cons-of-color-processing-with-machine-vision-cameras","status":"publish","type":"post","link":"https:\/\/www.adimec.com\/ja\/the-pros-and-cons-of-color-processing-with-machine-vision-cameras\/","title":{"rendered":"The Pros and Cons of Color Processing with Machine Vision Cameras"},"content":{"rendered":"

Often color processing is desirable when a user is viewing the images rather than a machine, such as with the outdoor applications of traffic monitoring or surveillance.\u00a0 Color processing can be done via the\u00a0frame grabber<\/a>\u00a0or right in the camera.\u00a0 This recent blog post on chipsight.com, which we have reposted below, provides some examples and the pros and cons of raw images versus color processed images.\u00a0\u00a0Adimec<\/a>\u00a0color cameras have\u00a0Bayer color filter arrays<\/a>\u00a0and can provide raw or RGB data to meet the needs of the specific application.<\/p>\n

http:\/\/chipsight.com\/raw-pixel-diet\/<\/a><\/p>\n

RAW Pixel Diet<\/strong><\/p>\n

by Craig Sullender on May 16, 2011<\/p>\n

What type of input does a vision application need? Can we use pixel data straight from the camera\u2019s image sensor? Or does the application need pixel data processed by an image sensor pipeline\u00a0(ISP)<\/a>\u00a0with color transformations and denoising?<\/p>\n

Computer vision:<\/em><\/strong>\u00a0Extract information from an image to perform a task.<\/p>\n

In other words a computer vision application may not need to produce an image for people to look at. This gives us a range of choices about the type of image sensor data we use for the vision input.<\/p>\n

The CMOS cameras under consideration (e.g. OV7690) contain an ISP, and output pixel color values in either a\u00a0Bayer color filter array pattern (CFA)<\/a>,\u00a0RGB<\/a>, or\u00a0YUV<\/a>\u00a0formats. The CFA pattern data output is called \u201craw,\u201d and is a data format, not a\u00a0file format like \u201c*.raw.\u201d<\/a><\/p>\n

Let\u2019s look at raw vs. RGB for the computer vision input.<\/p>\n

The image sensor array produces only a red, green, or blue value at each pixel location. Green occurs on every horizontal line, while red and blue are on alternating horizontal lines.<\/p>\n

\"Bayer<\/a><\/p>\n

The Bayer color filter array (or mosaic). Each sensor pixel location is covered by a red, green, or blue color filter. Image from\u00a0Wikipedia<\/a>.<\/p>\n

Post processing in the camera interpolates the missing color values with a\u00a0demosaicing<\/a>\u00a0algorithm, so that each pixel location then has three values, representing red, green, and blue intensities estimated for that location.<\/p>\n

The camera outputs the raw (CFA) or the interpolated RGB values according to user register settings. Our choices are RGB565 and Raw8.<\/p>\n

RGB565<\/a>\u00a0is packed into two 8-bit reads of the camera data bus: 5 bits red, 6 bits green, and 5 bits blue to make 16 bits for each pixel.<\/p>\n

Raw8 is one byte per pixel, and represents either red, green, or blue according to the pixel\u2019s location in the color filter array.<\/p>\n

RGB pros: complete red, green, and blue values for each pixel
\nRGB cons: only 5 bits resolution, twice the data rate, twice the data
\nRaw pros: Full 8 bits resolution, lower data rate
\nRaw cons: missing two color values at each pixel<\/p>\n

What are the trade-offs between RGB and raw for a computer vision processing step like motion detection by background removal?<\/p>\n

The background is an earlier image from the camera stored in a frame buffer (SRAM or DRAM) and updated according to rules we won\u2019t go into here. The current image from the camera is subtracted pixel by pixel from the background, with the pixels from the two sources aligned so that they correspond spatially. Pixel 1 of the current image is subtracted from pixel 1 of the background, then pixel 2 of the current image is subtracted from pixel 2 of the background, and so on. If nothing has changed over time in the view of the camera, then there are no moving objects, and the subtraction step results in values near zero, depending on noise. If something moved, a group (region) of pixels will show absolute values greater than a chosen threshold.<\/p>\n

Video example<\/a>\u00a0using successive frames (video images) as the background. Moving objects are displaced only a few pixels in distance compared to their location in the background frame, so the result is mostly edges. For our purpose the camera is in a fixed location and the background frame is maintained over time, so that moving objects will cause\u00a0larger groups of motion pixels.<\/a><\/p>\n

Grayscale images work fine for motion detection, but we are using color for greater sensitivity and for compatibility with other vision functions used in combination with the motion detection.<\/p>\n

With RGB you are subtracting red from red, green from green, and blue from blue. With raw you are missing two colors at every pixel location, so how can it work? It works for detecting changes in the scene because the camera is in a fixed position, so the color filter array (CFA) is in a fixed alignment with the scene. Though the colors are spatially distinct, there is still detection happening at each pixel. Objects that move will be detected by the raw format the same as the RGB.<\/p>\n

But not exactly. For example a scene with very little red in it will not detect objects at the red CFA locations with as much sensitivity as the other CFA locations. The detected region will have holes for every red location. To fill in the holes, you can filter the detection result by spreading motion pixels from neighboring locations. If the green to the left shows motion, then define the red location to its right as a motion pixel also.<\/p>\n

That works for filtering the results horizontally. What about vertically? For example if the blue CFA locations show motion, but not the red and green, you\u2019ll get a horizontally striped result. Filtering vertically is unfortunately a bit more demanding on resources. You have to save the current image line motion result in a line buffer so you can access the motion result for the pixel above the pixel you are filtering. Fortunately you only need to save one bit to indicate a pixel that changed due to motion.<\/p>\n

RGB pros: full color detection at each pixel location
\nRGB cons: lower sensitivity*, three subtractions for each pixel
\nRaw pros: 8 bit sensitivity, one subtraction per pixel
\nRaw cons: partial color, requires filtering<\/p>\n

* RGB565 sensitivity is arguably greater than 5 or 6 bits when you take into account the combined measurements of red, green, and blue.<\/p>\n

Which format you choose depends on how much memory is available for the background frame buffer, what data rate the vision input can handle, and how much sensitivity the application needs, compared to the added filtering required for raw.<\/p>\n","protected":false},"excerpt":{"rendered":"

Often color processing is desirable when a user is viewing the images rather than a machine, such as with the …<\/p>\n","protected":false},"author":3,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"content-type":"","_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"footnotes":""},"categories":[1],"tags":[],"class_list":["post-7612","post","type-post","status-publish","format-standard","hentry","category-uncategorized","category-1","description-off"],"yoast_head":"\nThe Pros and Cons of Color Processing with Machine Vision Cameras - Adimec JA<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.adimec.com\/ja\/the-pros-and-cons-of-color-processing-with-machine-vision-cameras\/\" \/>\n<meta property=\"og:locale\" content=\"ja_JP\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"The Pros and Cons of Color Processing with Machine Vision Cameras - Adimec JA\" \/>\n<meta property=\"og:description\" content=\"Often color processing is desirable when a user is viewing the images rather than a machine, such as with the …\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.adimec.com\/ja\/the-pros-and-cons-of-color-processing-with-machine-vision-cameras\/\" \/>\n<meta property=\"og:site_name\" content=\"Adimec JA\" \/>\n<meta property=\"article:published_time\" content=\"2011-06-08T08:22:18+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2018-07-27T15:14:04+00:00\" \/>\n<meta property=\"og:image\" content=\"\/\/cdn2.hubspot.net\/hub\/20421\/file-13402407-bmp\/images\/bayer-color-filter-array.bmp\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"\u57f7\u7b46\u8005\" \/>\n\t<meta name=\"twitter:data1\" content=\"\" \/>\n\t<meta name=\"twitter:label2\" content=\"\u63a8\u5b9a\u8aad\u307f\u53d6\u308a\u6642\u9593\" \/>\n\t<meta name=\"twitter:data2\" content=\"5\u5206\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.adimec.com\/ja\/the-pros-and-cons-of-color-processing-with-machine-vision-cameras\/\",\"url\":\"https:\/\/www.adimec.com\/ja\/the-pros-and-cons-of-color-processing-with-machine-vision-cameras\/\",\"name\":\"The Pros and Cons of Color Processing with Machine Vision Cameras - Adimec JA\",\"isPartOf\":{\"@id\":\"https:\/\/www.adimec.com\/ja\/#website\"},\"datePublished\":\"2011-06-08T08:22:18+00:00\",\"dateModified\":\"2018-07-27T15:14:04+00:00\",\"author\":{\"@id\":\"\"},\"breadcrumb\":{\"@id\":\"https:\/\/www.adimec.com\/ja\/the-pros-and-cons-of-color-processing-with-machine-vision-cameras\/#breadcrumb\"},\"inLanguage\":\"ja\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.adimec.com\/ja\/the-pros-and-cons-of-color-processing-with-machine-vision-cameras\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.adimec.com\/ja\/the-pros-and-cons-of-color-processing-with-machine-vision-cameras\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.adimec.com\/ja\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"The Pros and Cons of Color Processing with Machine Vision Cameras\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.adimec.com\/ja\/#website\",\"url\":\"https:\/\/www.adimec.com\/ja\/\",\"name\":\"Adimec JA\",\"description\":\"Reliable Industrial Cameras\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.adimec.com\/ja\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"ja\"},{\"@type\":\"Person\",\"@id\":\"\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"The Pros and Cons of Color Processing with Machine Vision Cameras - Adimec JA","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.adimec.com\/ja\/the-pros-and-cons-of-color-processing-with-machine-vision-cameras\/","og_locale":"ja_JP","og_type":"article","og_title":"The Pros and Cons of Color Processing with Machine Vision Cameras - Adimec JA","og_description":"Often color processing is desirable when a user is viewing the images rather than a machine, such as with the …","og_url":"https:\/\/www.adimec.com\/ja\/the-pros-and-cons-of-color-processing-with-machine-vision-cameras\/","og_site_name":"Adimec JA","article_published_time":"2011-06-08T08:22:18+00:00","article_modified_time":"2018-07-27T15:14:04+00:00","og_image":[{"url":"\/\/cdn2.hubspot.net\/hub\/20421\/file-13402407-bmp\/images\/bayer-color-filter-array.bmp","type":"","width":"","height":""}],"twitter_card":"summary_large_image","twitter_misc":{"\u57f7\u7b46\u8005":"","\u63a8\u5b9a\u8aad\u307f\u53d6\u308a\u6642\u9593":"5\u5206"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.adimec.com\/ja\/the-pros-and-cons-of-color-processing-with-machine-vision-cameras\/","url":"https:\/\/www.adimec.com\/ja\/the-pros-and-cons-of-color-processing-with-machine-vision-cameras\/","name":"The Pros and Cons of Color Processing with Machine Vision Cameras - Adimec JA","isPartOf":{"@id":"https:\/\/www.adimec.com\/ja\/#website"},"datePublished":"2011-06-08T08:22:18+00:00","dateModified":"2018-07-27T15:14:04+00:00","author":{"@id":""},"breadcrumb":{"@id":"https:\/\/www.adimec.com\/ja\/the-pros-and-cons-of-color-processing-with-machine-vision-cameras\/#breadcrumb"},"inLanguage":"ja","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.adimec.com\/ja\/the-pros-and-cons-of-color-processing-with-machine-vision-cameras\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.adimec.com\/ja\/the-pros-and-cons-of-color-processing-with-machine-vision-cameras\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.adimec.com\/ja\/"},{"@type":"ListItem","position":2,"name":"The Pros and Cons of Color Processing with Machine Vision Cameras"}]},{"@type":"WebSite","@id":"https:\/\/www.adimec.com\/ja\/#website","url":"https:\/\/www.adimec.com\/ja\/","name":"Adimec JA","description":"Reliable Industrial Cameras","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.adimec.com\/ja\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"ja"},{"@type":"Person","@id":""}]}},"_links":{"self":[{"href":"https:\/\/www.adimec.com\/ja\/wp-json\/wp\/v2\/posts\/7612","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.adimec.com\/ja\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.adimec.com\/ja\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.adimec.com\/ja\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/www.adimec.com\/ja\/wp-json\/wp\/v2\/comments?post=7612"}],"version-history":[{"count":10,"href":"https:\/\/www.adimec.com\/ja\/wp-json\/wp\/v2\/posts\/7612\/revisions"}],"predecessor-version":[{"id":11556,"href":"https:\/\/www.adimec.com\/ja\/wp-json\/wp\/v2\/posts\/7612\/revisions\/11556"}],"wp:attachment":[{"href":"https:\/\/www.adimec.com\/ja\/wp-json\/wp\/v2\/media?parent=7612"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.adimec.com\/ja\/wp-json\/wp\/v2\/categories?post=7612"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.adimec.com\/ja\/wp-json\/wp\/v2\/tags?post=7612"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}