{"id":4438,"date":"2023-03-22T10:33:08","date_gmt":"2023-03-22T14:33:08","guid":{"rendered":"https:\/\/blog.daed.com\/?p=4438"},"modified":"2024-07-11T10:59:14","modified_gmt":"2024-07-11T14:59:14","slug":"using-synthetic-data-to-drive-machine-learning","status":"publish","type":"post","link":"https:\/\/blog.daed.com\/?p=4438","title":{"rendered":"Using Synthetic Data to Drive Machine Learning"},"content":{"rendered":"

What is Machine Learning?\u00a0<\/span><\/strong><\/h2>\n

Over the past few years, data scientists have been making remarkable breakthroughs in the field of machine learning, a process that allows humans to quite literally teach computers how to recognize <\/span>things<\/span><\/i>. As children, humans learn to recognize and categorize <\/span>things<\/span><\/i> like trees, planes, and dogs by being exposed to dozens or even hundreds of examples of each in the wild. Over time and with guidance from the adults around us, w<\/span><\/span><\/span>\"Daedaluse learn to recognize what makes a dog different from a cat or a tree different from a bush and we start to associate names and functions with these <\/span>things <\/span><\/i>\u2014 that cylindrical ceramic container on your desk is a mug<\/em> that stores coffee; the annoying creature that wakes you up every morning is a bird<\/em>, and so on. Computers don\u2019t have the benefit of a childhood to learn these things, but machine learning mimics this process: a computer model is \u201ctrained\u201d on numerous examples of whatever it is you\u2019d like your computer to learn to recognize until it is able to figure out the rules that make things, well \u2026 things. Those things might range from <\/span><\/span>new galaxies in old astrological data<\/span><\/a> to <\/span>rare and dangerous diseases in patients.<\/span><\/a> The possibilities for machine learning to change the world are as limitless as human imagination.<\/span><\/span><\/p>\n

At Daedalus we are exploring new and innovative ways to incorporate machine learning into our designs. Currently, one of the most valuable applications of machine learning relates to computer vision, where models can be used to identify, categorize, and locate specific things in images and videos. For example, a home security system could be trained to alert its owners to when an unrecognized person is at the door. Or a computer for a fabrication line could be trained to detect faulty assemblies with nothing more than a webcam. Or a disaster recovery drone could be trained to<\/span> <\/span>identify survivors of natural disasters from aerial footage<\/span><\/a> more effectively than a human could. This type of model \u2014 ones that can be trained to find specific objects within an image \u2014 are called \u201cObject Detection\u201d models, and they are increasingly being used in all aspects of our lives.<\/span><\/span><\/p>\n

\"The<\/a>
Object Detection Model Output: Birds<\/figcaption><\/figure>\n

Machine Learning Challenges\u00a0<\/span><\/strong><\/h2>\n

One of the most difficult challenges to overcome in creating any kind of machine learning model is in finding thousands (or more) of examples of the thing that you want your model to learn to recognize. This training data<\/em> is increasingly becoming a vital resource in the tech industry and entire companies have formed just to collect and distribute these data. Just as it takes exposure to hundreds or thousands of examples for a human to learn what birds are or how particle physics works, machine learning models need to be trained on hundreds of thousands of data points. Though many organizations have worked to create publicly accessible datasets, at Daedalus we\u2019ve learned that these datasets aren\u2019t particularly useful for more specialized tasks. And while it is certainly <\/span>possible<\/span><\/i> to aggregate specialized datasets by hand, it would involve capturing thousands of images and manually labeling them with the information the model needs in order to learn. This isn\u2019t really feasible and certainly would not be very easy.\u00a0<\/span><\/span><\/p>\n

So at Daedalus, we\u2019ve tackled this problem by using a new type of data called synthetic data<\/em>. Instead of painstakingly photographing whatever we want our models to detect, we use 3D simulations to render photos that look close enough to our desired objects to allow for machine learning.<\/span><\/p>\n

While it may sound a bit surreal to think that a computer can learn what something is without ever actually seeing it, humans do something similar every day. We are easily able to recognize cartoon drawings or graphical icons and associate them with physical objects. For example, the pedestrian image that appears as an LED icon on traffic signals is made of basic shapes, but we are still able to recognize it as a person because of the baseline features. Synthetic data utilizes the idea of distinguishing baseline features to help a model extrapolate what it learns from simulations into real world applications.<\/span><\/p>\n

To accomplish this we use a tool called Blender that allows us to create highly detailed 3D models of whatever it is that we want our model to learn to recognize. Blender also has an entire scripting environment built in, which means that every single aspect of Blender can be automated, including the creation of full 3D scenes and the extraction of object locations within rendered images. This allows us to effortlessly generate a full list of context specific examples to feed directly into our model without any kind of human involvement. An entire dataset of localized objects can be created with just a click of a button.<\/span><\/p>\n

But relying on simulations to represent our objects\u2019 baseline features introduces another challenge. We have to ensure that our synthetic data is accurate enough to highlight the important features of the objects we want to detect, but still varied enough that our model doesn\u2019t inadvertently learn to only recognize Blender\u2019s simulated data. We achieve this through something called Domain Randomization, a process wherein we deliberately introduce variability into our synthetic images to help the model understand the various conditions in which our objects can be found.\u00a0<\/span><\/p>\n

We render our desired objects in front of different image backgrounds to place them in new environments. We can also create a full range of possible lighting conditions for our desired objects by varying light intensities, colors, and shapes. We can even add \u201cnoise\u201d objects that we don\u2019t wish our model to detect so that it can learn to avoid false positives. All of this variability helps the model to understand what is and isn\u2019t important for identifying objects. For example, a box is still a box regardless of how it is rotated, lit, or placed within a scene. Domain randomization allows us to create varied examples to show our model, including ones that would never be found in the real world. By randomizing our synthetic data we can effectively have our model treat real world conditions as yet another random variation in our domain.\u00a0<\/span><\/p>\n

\"Packages<\/a>
Packages rendered on different backgrounds and with irrelevant noise<\/figcaption><\/figure>\n

Ideally, our model will learn to recognize the desired object (the package) and to distinguish it from the background and other noise, as pictured below.<\/span> <\/span><\/p>\n

\"The<\/a>
Synthetic data generation goal, a mask differentiating the cardboard box pixels from the background pixels<\/figcaption><\/figure>\n

This technique is immensely powerful for applications in which little pre-existing training data exists. If we wanted to build an AI home security system that could send a notification to a user\u2019s phone when a package has been delivered, we\u2019d need thousands of photos of packages on doorsteps, with data entered by hand related to the pixel coordinates of each package, and in a format the model could understand. Collecting data like that manually would take days, if not months. To make matters more complicated, the ethics of how AI training data is collected is becoming more of a concern. Many homeowners would be very unhappy to discover if their home security system were recording pictures of packages on their doorsteps and sending them off to train a model without their consent. Synthetic data solves both of these problems, as the rendering software generates completely new and unique data along with all the necessary pixel locations for the model to learn from.<\/span><\/p>\n

Exciting Endeavors Ahead<\/span><\/strong><\/h2>\n

Synthetic data is already being used in a wide variety of technologies.<\/span> <\/span>Google\u2019s Waymo has been using simulations to train self-driving cars<\/span><\/a> without the need to rely on potentially unsafe prototypes being driven in the real world. Banks such as J.P. Morgan use synthetic data to <\/span>help detect fraudulent activity<\/span><\/a> without violating the privacy of customers or consumer protection laws by harvesting actual financial data. In some instances <\/span>models trained on synthetic data have been shown to perform better than models trained solely on real-world data<\/span><\/a>. By all indications synthetic data is going to become increasingly relevant as AI becomes a bigger part of our lives. At Daedalus we\u2019re very excited by the results we\u2019ve been getting from our synthetically trained models, and we can\u2019t wait to discover what we can accomplish next with this revolutionary new technique.<\/span><\/span><\/span><\/p>\n","protected":false},"excerpt":{"rendered":"

What is Machine Learning?\u00a0 Over the past few years, data scientists have been making remarkable breakthroughs in the field of machine learning, a process that allows humans to quite literally …<\/p>\n","protected":false},"author":10,"featured_media":4444,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3,8],"tags":[217,216,214,163,215],"class_list":["post-4438","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-product-development","category-products-culture","tag-blender","tag-domain-randomization","tag-machine-learning","tag-product-development","tag-synthetic-data"],"yoast_head":"\nUsing Synthetic Data to Drive Machine Learning - daed.com<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/blog.daed.com\/?p=4438\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Using Synthetic Data to Drive Machine Learning - daed.com\" \/>\n<meta property=\"og:description\" content=\"What is Machine Learning?\u00a0 Over the past few years, data scientists have been making remarkable breakthroughs in the field of machine learning, a process that allows humans to quite literally ...\" \/>\n<meta property=\"og:url\" content=\"https:\/\/blog.daed.com\/?p=4438\" \/>\n<meta property=\"og:site_name\" content=\"daed.com\" \/>\n<meta property=\"article:published_time\" content=\"2023-03-22T14:33:08+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2024-07-11T14:59:14+00:00\" \/>\n<meta property=\"og:image\" content=\"http:\/\/blog.daed.com\/wp-content\/uploads\/2023\/03\/ObjectDetectionModelOutput-BIrds-scaled.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"2560\" \/>\n\t<meta property=\"og:image:height\" content=\"1808\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Daedalus Team\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Daedalus Team\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"7 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/blog.daed.com\/?p=4438#article\",\"isPartOf\":{\"@id\":\"https:\/\/blog.daed.com\/?p=4438\"},\"author\":{\"name\":\"Daedalus Team\",\"@id\":\"https:\/\/blog.daed.com\/#\/schema\/person\/f08aac4ae63cbed0fffa61088919d15f\"},\"headline\":\"Using Synthetic Data to Drive Machine Learning\",\"datePublished\":\"2023-03-22T14:33:08+00:00\",\"dateModified\":\"2024-07-11T14:59:14+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/blog.daed.com\/?p=4438\"},\"wordCount\":1425,\"publisher\":{\"@id\":\"https:\/\/blog.daed.com\/#organization\"},\"keywords\":[\"Blender\",\"Domain Randomization\",\"Machine Learning\",\"product development\",\"Synthetic Data\"],\"articleSection\":[\"product development\",\"Products & Culture\"],\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/blog.daed.com\/?p=4438\",\"url\":\"https:\/\/blog.daed.com\/?p=4438\",\"name\":\"Using Synthetic Data to Drive Machine Learning - daed.com\",\"isPartOf\":{\"@id\":\"https:\/\/blog.daed.com\/#website\"},\"datePublished\":\"2023-03-22T14:33:08+00:00\",\"dateModified\":\"2024-07-11T14:59:14+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/blog.daed.com\/?p=4438#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/blog.daed.com\/?p=4438\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/blog.daed.com\/?p=4438#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/blog.daed.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Using Synthetic Data to Drive Machine Learning\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/blog.daed.com\/#website\",\"url\":\"https:\/\/blog.daed.com\/\",\"name\":\"daed.com\",\"description\":\"research, design, and engineering thinking\",\"publisher\":{\"@id\":\"https:\/\/blog.daed.com\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/blog.daed.com\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/blog.daed.com\/#organization\",\"name\":\"daed.com\",\"url\":\"https:\/\/blog.daed.com\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/blog.daed.com\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/blog.daed.com\/wp-content\/uploads\/2019\/10\/White_Daedalus.png\",\"contentUrl\":\"https:\/\/blog.daed.com\/wp-content\/uploads\/2019\/10\/White_Daedalus.png\",\"width\":5249,\"height\":745,\"caption\":\"daed.com\"},\"image\":{\"@id\":\"https:\/\/blog.daed.com\/#\/schema\/logo\/image\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\/\/blog.daed.com\/#\/schema\/person\/f08aac4ae63cbed0fffa61088919d15f\",\"name\":\"Daedalus Team\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/blog.daed.com\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/5d6de936556c9bdce42cd8b08658caf153a45451b66de4cda47d7ee8a3aaeb83?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/5d6de936556c9bdce42cd8b08658caf153a45451b66de4cda47d7ee8a3aaeb83?s=96&d=mm&r=g\",\"caption\":\"Daedalus Team\"},\"sameAs\":[\"http:\/\/www.daed.com\"],\"url\":\"https:\/\/blog.daed.com\/?author=10\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Using Synthetic Data to Drive Machine Learning - daed.com","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/blog.daed.com\/?p=4438","og_locale":"en_US","og_type":"article","og_title":"Using Synthetic Data to Drive Machine Learning - daed.com","og_description":"What is Machine Learning?\u00a0 Over the past few years, data scientists have been making remarkable breakthroughs in the field of machine learning, a process that allows humans to quite literally ...","og_url":"https:\/\/blog.daed.com\/?p=4438","og_site_name":"daed.com","article_published_time":"2023-03-22T14:33:08+00:00","article_modified_time":"2024-07-11T14:59:14+00:00","og_image":[{"width":2560,"height":1808,"url":"http:\/\/blog.daed.com\/wp-content\/uploads\/2023\/03\/ObjectDetectionModelOutput-BIrds-scaled.jpg","type":"image\/jpeg"}],"author":"Daedalus Team","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Daedalus Team","Est. reading time":"7 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/blog.daed.com\/?p=4438#article","isPartOf":{"@id":"https:\/\/blog.daed.com\/?p=4438"},"author":{"name":"Daedalus Team","@id":"https:\/\/blog.daed.com\/#\/schema\/person\/f08aac4ae63cbed0fffa61088919d15f"},"headline":"Using Synthetic Data to Drive Machine Learning","datePublished":"2023-03-22T14:33:08+00:00","dateModified":"2024-07-11T14:59:14+00:00","mainEntityOfPage":{"@id":"https:\/\/blog.daed.com\/?p=4438"},"wordCount":1425,"publisher":{"@id":"https:\/\/blog.daed.com\/#organization"},"keywords":["Blender","Domain Randomization","Machine Learning","product development","Synthetic Data"],"articleSection":["product development","Products & Culture"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/blog.daed.com\/?p=4438","url":"https:\/\/blog.daed.com\/?p=4438","name":"Using Synthetic Data to Drive Machine Learning - daed.com","isPartOf":{"@id":"https:\/\/blog.daed.com\/#website"},"datePublished":"2023-03-22T14:33:08+00:00","dateModified":"2024-07-11T14:59:14+00:00","breadcrumb":{"@id":"https:\/\/blog.daed.com\/?p=4438#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/blog.daed.com\/?p=4438"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/blog.daed.com\/?p=4438#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/blog.daed.com\/"},{"@type":"ListItem","position":2,"name":"Using Synthetic Data to Drive Machine Learning"}]},{"@type":"WebSite","@id":"https:\/\/blog.daed.com\/#website","url":"https:\/\/blog.daed.com\/","name":"daed.com","description":"research, design, and engineering thinking","publisher":{"@id":"https:\/\/blog.daed.com\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/blog.daed.com\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/blog.daed.com\/#organization","name":"daed.com","url":"https:\/\/blog.daed.com\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/blog.daed.com\/#\/schema\/logo\/image\/","url":"https:\/\/blog.daed.com\/wp-content\/uploads\/2019\/10\/White_Daedalus.png","contentUrl":"https:\/\/blog.daed.com\/wp-content\/uploads\/2019\/10\/White_Daedalus.png","width":5249,"height":745,"caption":"daed.com"},"image":{"@id":"https:\/\/blog.daed.com\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/blog.daed.com\/#\/schema\/person\/f08aac4ae63cbed0fffa61088919d15f","name":"Daedalus Team","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/blog.daed.com\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/5d6de936556c9bdce42cd8b08658caf153a45451b66de4cda47d7ee8a3aaeb83?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/5d6de936556c9bdce42cd8b08658caf153a45451b66de4cda47d7ee8a3aaeb83?s=96&d=mm&r=g","caption":"Daedalus Team"},"sameAs":["http:\/\/www.daed.com"],"url":"https:\/\/blog.daed.com\/?author=10"}]}},"_links":{"self":[{"href":"https:\/\/blog.daed.com\/index.php?rest_route=\/wp\/v2\/posts\/4438","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog.daed.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.daed.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.daed.com\/index.php?rest_route=\/wp\/v2\/users\/10"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.daed.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=4438"}],"version-history":[{"count":2,"href":"https:\/\/blog.daed.com\/index.php?rest_route=\/wp\/v2\/posts\/4438\/revisions"}],"predecessor-version":[{"id":4455,"href":"https:\/\/blog.daed.com\/index.php?rest_route=\/wp\/v2\/posts\/4438\/revisions\/4455"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/blog.daed.com\/index.php?rest_route=\/wp\/v2\/media\/4444"}],"wp:attachment":[{"href":"https:\/\/blog.daed.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=4438"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.daed.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=4438"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.daed.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=4438"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}