Structured Vision in Manufacturing
https://commons.wikimedia.org/wiki/File:Factory_Automation_Robotics_Palettizing_Bread.jpg
Smart Manufacturing Engineering April 2019
By Bruce Morey
Providers are delivering varied sensors to match the needs and the budgets of industry users. As prices drop and capability grows, the question is what is the next best application?
Structured light systems measure surfaces by projecting a pattern of fringes, then using cameras and sophisticated software to convert them into point clouds of metrology data. Accuracy can reach the single-digit microns over millions of points. Capture rates on some newer equipment are measured in milliseconds. Affordability and choice are growing among these sensors. While most remain two-camera stereo set-ups, some are using single camera set-ups for more flexible deployment, especially near production lines. More deployments and more applications are the hallmarks of the industry.
One of the pioneers in the field was GOM, a company now expanding the options in its ATOS line of structured light systems. “In the past, everybody looked at ATOS sensors as the high-end but expensive scanner. But now [GOM] offers an entire line of structured-light scanners,” said Frank Stone national sales engineering manager for Capture 3D Inc., Santa Ana, Calif., a value-added reseller of ATOS sensors. “They recognized that to expand the market you’ve got to deliver to a wide variety of customers—not everybody needs as tight tolerances and accuracies as others
do,” he said.
A good example of how the company offers variety is its ATOS Core, an entry-level system. It
can grow in capability along with the needs of the owner. Entry-level applications such as reverse engineering or rapid prototyping, according to Capture3D, need scanning without advanced inspection functionality. A complete “entry level” kit includes software that converts point cloud data into STL formatted polygonal meshes. The Core offers an upgrade path from manual operation to semi-automated operation that could include adding touch probes and photogrammetry for higher quality metrology inspections. The same device can also be ultimately upgraded to full automation for industrial quality control inspections.
But GOM has not stopped improving its high-end systems either. Its latest are the ATOS 5 and ATOS 5X offerings, designed for industrial use and integration of measurements into production lines. One of the keys to using structured light in harsh environments is to shorten the collection time, making it less sensitive to vibration and noise. According to the company, the ATOS 5 captures scans in 0.2 seconds (200 milliseconds) at 100 frames per second. Combined with the ATOS Scanbox automation system, it could be used to measure automotive body closures, instrument panels, engine turbine blades or cast parts. Mounted on robots in custom integrations, it can measure complete body-in-whites, which are common in automotive manufacturing.
Variety in Projection Sources
To expand utility even further, the more advanced ATOS 5X uses a much brighter light source. “The ATOS 5X uses an integrated laser light compressor to provide that brighter light source, which expands the measurement volume,” explained Stone. “Where before we could only measure 700 mm, with this new source we can now measure up to 1,000 mm in the same time. It takes only 0.2 seconds per scan.” Technically, this is still structured light and does not use the coherence available in a laser to do interferometry—it is a very bright but still standard structured light device.
Hexagon Manufacturing Intelligence, North Kingstown, R.I., has also developed in recent years a wide range of structured light scanning devices. They are grouped in two basic technologies, those that use single slide projections and those that use multiple fringe projections. Single slide sensors emphasize speed and use in harsh conditions and include Hexagon’s WLS qFLASH, the BLAZE 600 and WLS 400.
“These sensors emphasize fast image acquisition,” said Amir Grinboim, technical program manager for Hexagon. These collect data on average in 0.015 seconds, or 15 milliseconds. The speed of collection is important—it is what makes them ideal in factory shop floor settings. “The philosophy is ‘measure wherever you want’,” said Grinboim.
The multiple fringe projection technology, on the other hand, is slower but with better resolution. These include the AICON offerings from Hexagon. “When we take a shot, the image is built out of a sequence of projections. And those projections are lines that are projected on the part, and those lines are moved and change orientation along with the image acquisition process. That is a relatively long process,” explained Grinboim, taking on average 2 seconds for a single collection. “The goal is not speed but the quality of the point cloud.”
Hexagon also uses a Red/Green/Blue (RGB) DLP projector in its newest sensor, the AICON StereoScan neo. The sensor also has a wide range of measurement fields, from 75 mm up to 1,100 mm. This is done through changing the camera lenses or the base length. The acquisition time is about 2 seconds and the device can be equipped with either an 8- or 16-megapixel camera.
What the RGB DLP projector is ideal for are applications where the measurements can be converted into a color map of deviations when compared to CAD nominals. These are then projected back onto the part that was just measured.
“[This is] ideal for design work, reworking for tooling, and showing overstock or understock material in casting. All of this can be visible to the engineer right on the part, without reference to a computer screen or a PDF,” explained Grinboim.
Hexagon calls this See What You Measure, or SWYM. “This is ideal in measuring and working with castings, forgings or injection plastics. If you have a large geometry part, our software will automatically stitch multiple collections together without the need for reference targets on the part,” he said.
Industrialization and Automation
“Popular applications [for structured light scanners] include legacy tooling for reverse engineering and aerospace applications. Both commonly require higher precision tolerances that cater to the structured light hardware,” explained Greg Groth, division manager for Exact Metrology, Brookfield, Wis. Exact Metrology offers contract measurement and scanning services, metrology equipment solutions, as well as hardware and software rentals. It offers or uses metrology devices ranging from Portable Arm CMMs to high-end CT scanners.
The company also uses GOM ATOS structured light systems. These include the ATOS II Triple Scan and ATOS III Triple scan, which Exact uses in its contract work. Exact also distributes the AICON PrimeScan from Hexagon Manufacturing Intelligence.
Automation is becoming a mainstream use of structured light devices. “Currently, the automation world is adopting the [structured light] technology. As robot cells become safer and more cost-effective in parallel to the hardware becoming cheaper, in-line automation popularity is rising,” Groth said. “As acquisition speeds increase to subsecond, that opens up a new world of throughput efficiencies.”
He also believes current industrial buzzwords of the day—such as AI, Internet of Things and Big Data Analytics—are having an effect on the adoption of structured light systems. “One big change is the dynamic feedback, or a looped system. For example, scanning the component, comparing to the designed intent CAD data, feeding back those discrepancies to the manufacturing process, and making the adaptive changes to the tooling to make correct parts,” he said.
Ian Scribner, portable 3D scanning product sales manager for Carl Zeiss Industrial Metrology LLC, Brighton, Mich., echoed this focus on automation in his observations. “[Automation is] where most of the focus has been over the past couple of years in enhancing [our] software in order to automate the hardware,” he explained.
The history of the structured light systems Zeiss offers also traces its roots to one of the early founders of the field, the Steinbichler company. Zeiss acquired the company and its technology a few years ago. The Zeiss COMET line of structured light systems comes from this heritage but is now showing some improvements unique to Zeiss. One example is the Zeiss ABIS II system used for detection of surface defects. It is especially useful in measuring Class A body surfaces in automotive manufacturing—again designed for use at or near the production line.
Another of Zeiss’ more recent offerings is the COMET Pro AE. The device is used in the company’s pre-packaged automation solutions, AIBox and AIBox flex systems. It also illustrates another trend prevalent in structured light systems by all providers—combining different metrology technologies to improve automation and accuracy.
“The COMET Pro AE is strictly dedicated to automation. It has a couple additional features—including photogrammetry,” said Scribner. “This helps capture larger components faster and with better accuracy. It also has light rings, which allow users to be able to scan sheet metal and provide better feature detail on holes or strops or cutouts of the sheet metal.”
Collection times of the COMET Pro AE are dependent on the size of the part. A sheet metal car body, for example, takes a few minutes. The result is tens of millions of points at 25 to 35 µm accuracy, according to Scribner. In fact, his biggest success stories in recent history have been in automating measurement of sheet metal parts.
While the COMET Pro AE is offered as an “at-line” device useful for sample lot inspections, Zeiss also offers a structured light solution for in-line inspection with its AIMax Inline and BestFit offerings. The company advertises these as good for quality assurance, location recognition and production-control jobs, including robot guidance.
The device can also output scan points converted into STL formatted surfaces. “Often, the COMET AIMax is programmed to look for specific features for doing in-line gaging,” he said. Since it is programmable, it avoids the pitfalls of hard gaging, at an increasingly affordable price.
“Every year we have seen structured light systems as a group come down in price and go up in capability, opening new applications,” Scribner said. The compact size of the BestFit is enabled by its single-camera technology. Also, recognizing the harsh environment they are designed for, both sensors exhibit high-temperature stability through active compensation.
Affordability and Applications
Big changes are coming to the vision systems market, according to Scott Green, director of software for 3D Systems Inc., Rock Hill, S.C. “There is going to be a huge shift in the market for high-resolution structured light systems,” he said.
The company offers a broad range of products, from 3D printers to haptic force-feedback
devices, including a line of structured light metrology systems. The shift in structured light systems will come about because of dropping device prices even while quality and scanning coverage increases, according to Green.
“These are [systems] that can be soon bought on a credit card. Today, even, there are very low-cost, high-quality structured light sensors that sell for less than $10,000,” he said—a far cry from the original $100,000 plus systems originally introduced into the market. 3D Systems offers what it terms an industrial-grade scanner—the Capture and a smaller version, the Capture Mini. Another hand-held device also uses structured light for quick measurements of small items.
How is this possible? These systems need high quality and precision light sources and precise projection patterns. The cameras need to capture sharp images up to 16 megapixels in size. There is much-sophisticated software and mathematics that go into getting the data out of those components. Decades of refinement is making the software and algorithms more sophisticated. Yet, he maintains the systems are in essence … simple. “A structured light system is basically two cameras and a projector,” he said, with a computer and software.
The general consumer market is driving the cost and quality curve in those components. Computing at this level is becoming almost free. Mass produced smartphones and consumer-grade cameras mean digital cameras and light sources are advancing rapidly.
“The functional components, the actual components of building a structured light sensor are becoming much more affordable,” Green said. “So you see a downward shift in the cost of ownership for that kind of high-quality structured light as the components inside it become more readily available and cheap.”
Once the cost of hardware is driven down, he said price pressure on the accompanying software will increase. With total cost of ownership decreasing, that opens up using the now inexpensive structured light systems in dedicated uses, according to Green. Think measuring gap-and-flush in automotive body-in-white or 100 percent inspection of individual features in a machining cell by individual sensors.
“For the right applications, we will have much smarter sensors that are much closer to the place of production,” he said. “The role in desktop software for those applications will be much more like an information aggregator.”
