In recent years we have seen an explosion of the use of reality capture by traditional CAD and building information modeling companies. Its ability to capture existing conditions and create 3D models out of photographs or laser scans, without seams makes it a great fit for the AEC industry. Technologies used include 3D laser scanning, mobile and aerial LiDAR, and photogrammetry. This resulting point data is reconstructed into a 3D model. While accurate, LiDAR only captures 3D data leaving color out. Photogrammetry uses photographs to reconstruct a 3D image.
ReCap Enhancements
The interest in this technology has also expanded to geospatial offering efficient ways of being able to capture data for ground truth than traditional methods.
There are other contributing factors to this explosion, one being the cloud and the ease with which 3D point clouds can now be imported into CAD or BIM softwares. At one time this was a very laborious process and required tremendous bandwidth. It’s also possible to collect 3D scan data using a smartphone.
Some of the services offered around Reality Capture may include georeferencing, image registration, automatic calibration, calculating a polygon mesh, coloring, texture, parallel projections, DSM, simplification, scaling, filtration, coordinate system conversion, smoothing, measurement, inspection, and methods of imports and exports. Both camera images and laser scans can be combined.
Not only does reality capture offer a better way to produce designs or inspect existing structures, but it also provides new business opportunities.
In response to our questionnaire we heard from the providers of CAD and BIM software and how they have built applications to import 3D point cloud objects into their software where those objects can be further utilized. Some organizations have applications that can produce very large 3D models from point clouds or photographs, even for the purpose of designing 3D cities.
What are your products for Reality Capture and their features?
Engineering and surveying professionals use Bentley’s 3D imaging and point-cloud processing solutions to capture, process, and reuse existing site condition data. The software streamlines capture-to-model workflows, eliminates time-consuming translations, and accelerates image and point-cloud data performance to add accurate, real-world imagery to infrastructure projects.
ContextCapture is Bentley’s reality modeling application for producing large and challenging 3D models, including scales as large as entire cities, from simple photographs and/or point clouds. The 3D reality models can quickly and easily provide users with accurate digital context for design, construction, and operations decisions for all types of infrastructure projects throughout the world. Depending on the users’ project and organizational needs, ContextCapture is available as an on-premise or cloud processing service, allowing users to fast-track their projects by utilizing parallel processing and multiple desktops and servers.
With Descartes, users can integrate and process reality modeling data, such as reality meshes, point clouds, scalable terrain models, and raster data, for use in information modeling workflows to increase project quality. It uses high-fidelity imaging capabilities to support precision mapping and engineering, gives users a more complete integrated view of all information, and streamlines the production of deliverables in most standard industry formats for use in other applications.
Users can streamline scan-to-model workflows by importing point clouds from all major scanner manufacturers with Bentley’s Pointools., The application enables intuitive cleanup and preparation of point clouds for easier reuse in downstream applications.
Users can enliven their 3D reality models with LumenRT by adding life and nature and produce attention-grabbing visuals for sharing with other stakeholders and clients. – Chintana Herrin, Product Marketing Manager, Reality Modeling, Bentley Systems
Autodesk provides end-to-end, scan-to-BIM solutions through our AEC Collection, which includes our reality capture software, ReCap Pro.
ReCap Pro pairs with the Leica BLK360 imaging laser scanner through an iPad. Customers can user ReCap Pro for mobile to control the BLK360 and view, markup, measure, and collaborate on point clouds in the office or on the jobsite.
ReCap Photo is our newest desktop and cloud solution built for UAV and drone processes, offering expanded photogrammetry capabilities. The ReCap Pro for mobile app and ReCap Photo are both included in a subscription to ReCap Pro. Michael Mizuno, Senior Product Manager, ReCap Product Line, Autodesk
Reality Capture isn’t new to Vectorworks. In 2015, we released point cloud support, which allowed architects to employ their preferred 3D scanning device to capture a physical location and import a 3D point cloud object into Vectorworks software to manipulate and measure.
Last month, and with the help of our research, mobile and cloud teams, we released a new photogrammetry technology called Photos to 3D Model for our Vectorworks Cloud Portal and Vectorworks Nomad app. This feature is exclusively available to Vectorworks Service Select members. The Photos to 3D Model capability leverages the latest photogrammetry technologies with the processing power of Vectorworks Cloud Services to generate 3D models from digital photographs. –Steve Johnson, vice president of product development at Vectorworks, Inc.
IMAGINiT is partners with Leica Geosystems, Pix4D, and others. We represent, educate, and support the full line of Leica HDS laser scanning products, Pix4D applications, as well as Leica’s iCon series of robotic total stations to round trip reality capture data back into the physical world. –Jeff Bowers, Solutions Consultant, Reality Capture Solutions at IMAGINiT
What are some of the best applications of this technology?
- Combining drone photogrammetry with laser scanning for superior accuracy.
- Validating construction against design intent.
- Increase safety – drones have made it easier to safely capture hard-to-access or inaccessible areas where traditional surveying could be dangerous or impractical. Data captured by drone can be registered into virtual models in ReCap for further design in Autodesk design products such as AutoCAD, Revit, Infraworks, and Navisworks.
Early detection of problems through reality capture in the pre-design phase can prevent costly rework downstream. – Michael Mizuno, Senior Product Manager, ReCap Product Line, Autodesk
Photos to 3D Model will be a major asset when documenting existing site conditions. For instance, you can take a photo of an existing structure and then add to or alter it without having to model the structure from scratch. This feature really shines when you are looking to capture structures like buildings that require complex detailed work in the façade, or older buildings that are not perfectly square. It’s also much easier to get dimensions from photos, rather than with a tape measure. Steve Johnson, vice president of product development at Vectorworks, Inc.
Different technologies are best deployed for different project purposes. Overall, the digitization of the existing conditions allows for the project design team to be better informed about the existing design constraints, validate any assumptions that may have been made from available as-built drawings, monitor and analyze changing conditions by comparing highly detailed snapshots in time, and better communicate design intent with highly detailed visualizations of proposed items in context of the existing conditions. Jeff Bowers, Solutions Consultant, Reality Capture Solutions at IMAGINiT
ContextCapture can be applied to many industries and asset types. It provides context for design, construction, and operations decisions for all types of infrastructure projects. Here is a sampling of user stories that credited reality modeling in their infrastructure projects: User Project Profiles page on Bentley’s website. Chintana Herrin, Product Marketing Manager, Reality Modeling, Bentley Systems
Some technologies allow you to take photos with a smartphone and create 3D models out of the resulting photos. Is this something your company does and if so, describe that process.
Certainly! That process is possible and, when deployed correctly, offers a very low-cost solution to specific problems. Photogrammetric processes are not dependent on platform…they simply digest any photos or video we feed it. The user has to ensure that adequate overlap of multiple photos and focal point consideration principles are being used to ensure an acceptable result. The videos are broken down to photos by frames-per-second and are sure to maintain great photo to photo overlap, but may be significantly heavier to process. Photogrammetric processes, however, typically will not automate the considerations of location, orientation, or scale, so the user has to ensure there is some form of reference geometry available. Additionally, the quality of the photos completely drives the quality of the output. We need to have realistic expectations of the quality of the resulting data if we choose to use low sophistication collection equipment. Jeff Bowers, Solutions Consultant, Reality Capture Solutions at IMAGINiT
Yes, photo-to-point cloud creation from smartphones, SLR cameras, and GoPro cameras is possible in ReCap.
Build Change, a nonprofit specializing in natural disaster recovery and resilience, set out to retrofit homes in Nepal for earthquake recovery. They assembled 3D models of the homes in ReCap by collecting mobile phone photos from the community. – Michael Mizuno, Senior Product Manager, ReCap Product Line, Autodesk
Bentley’s ContextCapture Mobile app is an easy-to-use reality modeling application that automatically generates 3D models with photos taken from your smartphone or tablet using ContextCapture Cloud Processing Service. You can share the 3D mesh on the device, store it by using ProjectWise ContextShare, and/or share the model link with your collaborators via email. Chintana Herrin, Product Marketing Manager, Reality Modeling, Bentley Systems
Vectorworks
This is exactly what we are doing. With Photos to 3D Model, you are able to use the Nomad app on your phone or mobile device to take photos of a building, then send those photos to Vectorworks Cloud Services. From there, the power of cloud processing makes a point cloud and textured mesh out of the images. Next, you can bring the generated point cloud or textured mesh into your Vectorworks model. We provide step-by-step instructions that include best practices to steer you through the process of taking quality photos. Steve Johnson, vice president of product development at Vectorworks, Inc.
AI_14_Reality Modeling Empowers Governments with Digital Cities and to Explore the Future for Autonomous Vehicles (5)
3D reality model of the existing urban environment in busy Santa Clara County, California
Image Courtesy of The Sanborn Map Company
(The Sanborn Map Company, Sanborn3D HD Maps for Autonomous Driving: Santa Clara, Santa Clara County, California, USA)
What sets apart the solutions that you offer for Reality Capture?
ContextCapture is an end-to-end reality modeling solution. From capture to consumption, it ensures interoperability so users can easily share and stream the 3D models into various applications to provide digital context, accelerating the decision-making process during all phases of an asset lifecycle with a connected data environment.
ContextCapture helps dramatically compress timelines with its high-end scalability by speeding the processing of 3D models. Its inherent high performance parallel computing power leverages the latest cloud technology to rapidly provide digital context for all projects.
With ContextCapture, users can generate high-fidelity, 3D engineering-ready reality meshes with multi-resolution inputs. It can incorporate the best of both worlds – the versatility and convenience of high-resolution photography and, where needed, the additional accuracy of point clouds for use in design, construction, and operational decisions. Chintana Herrin, Product Marketing Manager, Reality Modeling, Bentley Systems
Currently, many of the available technologies like LiDAR or laser scans are extremely expensive and cost several thousand dollars. With Photos to 3D Model, architects can capture data with inexpensive technology that’s already in their pockets. Since the new feature will be integrated in Vectorworks and available to all Vectorworks Service Select members, there are no additional costs. Steve Johnson, vice president of product development at Vectorworks, Inc.
- Ease of use – ReCap was designed so that anyone can use it, whether an experienced professional or a novice.
- Fast processing, highly accurate data.
- Hardware agnostic – ReCap Pro offers a complete solution, allowing the integration of laser scan and UAV scan data for accurate, 3D modeling. ReCap Pro pairs with most hardware.
- On mobile – You can access cloud-powered reality data across multiple platforms – web, desktop app, and in the field with your iPad.
- Media & Entertainment – you can convert Reality Capture data into 3D mesh models for media and entertainment applications.
Affordable pricing with flexible monthly or annual subscription options. See pricing.- Michael Mizuno, Senior Product Manager, ReCap Product Line, Autodesk
IMAGINiT’s role in the industry is center focused on the education, consulting, and implementation of these technologies into our client’s project workflow processes. While our clients design solutions to their given project types, we design highly efficient processes and workflows to ensure their technology investments are highly utilized and their staff is well educated in putting those investments towards profits. – Jeff Bowers, Solutions Consultant, Reality Capture Solutions at IMAGINiT
Can you describe a successful workflow for using Reality Capture?
A successful workflow using Reality Capture lets you introduce real-world data into your design process. Capturing existing site conditions with a laser scanning workflow provides highly accurate 3D model data. This data can be imported into Vectorworks where accurate measurements can be taken to support the design or remodeling process. The data can be used to create a virtual model of reality, or it can be used to compare existing conditions with the proposed design.
With a Photos to 3D Model workflow, the shape and color can be captured and added to the virtual model with a much simpler workflow. 3D model data from photos can be used to achieve similar outcomes as compared with expensive and time-consuming laser-scanning workflows. Simply take photos with your phone or camera. Vectorworks Cloud Services processes these 2D images and generates 3D model data that can be imported directly into Vectorworks. Data at your fingertips. See it. Capture it. Have fun! Steve Johnson, vice president of product development at Vectorworks, Inc.
Capture existing site conditions using hardware such laser scanners or camera-mounted UAVs. The detailed set of data or “points” is then processed in ReCap and scan data is automatically registered, stitched into a coordinate system, cleaned up, and analyzed. The final product is a point cloud or mesh ready for CAD and BIM authoring tools.
For example, in a building project, you can bring ReCap data into Revit to create a 3D model or floor plan. In Revit, you could add elevation and dimension notes, create window objects, and place gridline annotations. There’s seamless and consistent interoperability across Autodesk products.
You can overlay the 3D model with real-world photos to create interesting visualizations for staging or quick snapshots of your project. – Michael Mizuno, Senior Product Manager, ReCap Product Line, Autodesk
Workflows can vary wildly depending on the reality capture technology being deployed and the final resulting deliverable that we intend to create. That being said, a few overall principles govern. First, we need to deploy the most efficient field data collection technique that will serve the requirements of the given project. Secondly, we need to initially process that data as quickly as possible. Automation takes care of the vast majority of the heavy lifting, but we need to maintain our abilities to verify the accuracy of the data. Lastly, we need to be able to use the geometry of the data to automate the creation of our models. Basically, if I must trace the point cloud to create my models, I’m doing it wrong. Tracing introduces too much human error and takes too much time from my project budget. – Jeff Bowers, Solutions Consultant, Reality Capture Solutions at IMAGINiT
A successful reality modeling project is the Development of Anderson Road Quarry Site in Hong Kong project by CEDD-AECOM-The Earth Solutions. The team used a capture-to-consumption workflow and leveraged drone technology. For details, view the project case study: Development of Anderson Road Quarry Site in Hong Kong project by CEDD-AECOM-The Earth Solutions. – Chintana Herrin, Product Marketing Manager, Reality Modeling, Bentley Systems
How do you feel Reality Capture has revolutionized the AEC industry?
Reality capture bridges the gap between the perfection of the design environment with the imperfect reality of the real world. By creating a handshake between these environments, designs are more accurate, more constructible, and projects are more likely to see completion on time and on budget. -Jeff Bowers, Solutions Consultant, Reality Capture Solutions at IMAGINiT
Our free eBook details how Reality Capture is changing the design and construction industry, and 7 ways it can help deliver projects with more accuracy and efficiency.
Reality capture is improving accuracy, increasing safety, accelerating efficiency through rapid data collection, reducing tedious manual work, and helps validate construction against design intent. – Michael Mizuno, Senior Product Manager, ReCap Product Line, Autodesk
Reality Capture is rapidly becoming a cheaper, more accessible and automated way to create a model. It makes designing and redesigning much easier, because you can effortlessly compare the virtual and the real. When combined with clash detection capabilities, you can capture a rough shape and compare it to your 3D model to see how items fit into spaces.
One of the most exciting advances in Reality Capture is the key role of drones in the process, which help to easily create 3D models to see how the construction phase of a project is coming together each day. The Photos to 3D Model solution handles photos from drones extremely well. This will not only benefit the AEC industry, but also the landscape architecture, landscape design and urban planning industries. Specifically, this is evident in the site design tools that Vectorworks offers. –Steve Johnson, vice president of product development at Vectorworks, Inc.
Reality capture has completely revolutionized the AEC industry because it can significantly reduce risks and costs, as well as save time on infrastructure projects. Reality modeling is going mainstream, as many organizations are recognizing its benefits. In 2016, 62 Be Inspired Awards nominated projects credited reality modeling and, in 2017, that number grew to 111 from 36 countries. -Chintana Herrin, Product Marketing Manager, Reality Modeling, Bentley Systems
What improvements in Reality Capture would you like to see and/or are working on in your research and development?
We are now working on a visual operations interface. It will combine reality models with asset information, allowing an asset manager and field technicians to access asset intelligence through their 3D location with context. Additionally, artificial intelligence (AI) technologies are becoming mainstream across the board in every industry. These technologies include deep machine learning, computer vision, and optimization for advancing infrastructure through semantic reality modeling. These uses include segmenting 3D reality models for smart city, classifying images and videos for semantic 3D model, and detecting and quantifying defects for infrastructure condition assessment. –Chintana Herrin, Product Marketing Manager, Reality Modeling, Bentley Systems
As hardware solutions become more powerful and portable, we’ll also see reality capture software evolve. The Autodesk ReCap team continues our mission in democratizing reality capture through simplifying and automating processes and working with our partners to create next-gen solutions. –Michael Mizuno
Senior Product Manager, ReCap Product Line, Autodesk
I would like to see design softwares enhance their ability to use the point cloud in their designs, thus negating the need to create an existing conditions model from the point cloud…just use the point cloud itself! –Jeff Bowers, Solutions Consultant, Reality Capture Solutions at IMAGINiT
At Vectorworks, we look forward to continued improvements in solutions that will incorporate reality into the design process. We look forward to reduced costs, easier access and ease of use when working with highly accurate and dense laser-scanned data. We are also enthusiastic about the increased ease of use and improved accuracy of our new Photos to 3D Model technology. We are even using Reality Capture to help us improve the augmented reality in Nomad. It’s a new way of thinking — the opposite of AR — you are augmenting the virtual world with reality. The future will certainly continue to provide technological advances that support the integration between the virtual model and our real world. At Vectorworks, we are always working to explore these new technologies that will continue to inform, simplify and augment the design process.
–Steve Johnson, vice president of product development at Vectorworks, Inc.
Customer stories include the following:
1. Project: High altitude Scan-to-BIM
Description: A small firm’s scan-to-BIM success story using Autodesk ReCap and Revit for a hospital renovation project in the French alps.
- Project: Capturing Volterra’s Historic Preservation “
Description: Using Faro laser scanners, 3DR drones, and Autodesk software for the digital preservation of the historic preservation of the ancient city of Volterra, Italy.
Description: Autodesk Forge makes it possible to bring drone-collected data to the cloud, where ReCap reality capture software stitches the information into accurate orthomosaics, point clouds, 3D meshes, and more.
- Project: WeWork’s first look at the BLK360 and ReCap Pro for mobile solution
“If our designers in the field will use ReCap (for) mobile, then they can hand us processed scans instead of hundreds of scans we now have to process, they’ll be ready right then and that means we can do more scanning. We can up the volume so we can scan multiple times over the course of a project instead of just pre-construction.”
–Thad Wester, Senior Reality Capture Specialist, WeWork
Michael Mizuno, Senior Product Manager, ReCap Product Line, Autodesk
From IMAGINiT, comes the story Olsson Associates Scanning Horizons for Opportunity
“On a recent project, I used the Photos to 3D Model feature of Vectorworks to greatly enhance the project flow. 50 photos taken in about 5 minutes allowed me to generate a point cloud accurate enough to ensure our equipment would fit in the 4,700 square foot complicated industrial space, including multiple overhead ducts and conduits, all while never leaving the ground. To achieve similar accuracy using conventional measuring methods would have taken at least a day and been less accurate. I intend to use Photos to 3D Model on all of our future projects in existing facilities.” –Patrick Stanford, vice president of engineering at Rochem Americas, Inc.
“It is indeed quite an awesome ‘drop-to-model’ functionality: take a few pictures in your smart phone, drop them in the Cloud portal, and you get an instant model of your building!” -Carlotta Birelli, CAD manager for Kuehn Malvezzi, a German-Italian architecture firm
These reality modeling user projects highlight our users’ extraordinary work, sharing innovative practices in reality capture and visualization to solve design, engineering, construction, and operations challenges. To view the projects, visit the User Project Profiles page on Bentley’s website. – Chintana Herrin, Product Marketing Manager, Reality Modeling, Bentley Systems.
Are your Reality Capture technologies used more in certain countries than in others? Please describe interest and why you think certain locations are using more Reality Capture.
In 2017, 111 Be Inspired Awards projects from 36 countries credited reality modeling for helping develop and complete their projects. Increasing urbanization is driving the need for more detailed and up-to-date digital models of entire cities. ContextCapture leads the field in its ability to scale to very large city-wide data capture and processing to deliver high fidelity digital city models. These models, built from aerial photography, can be linked with existing city data, GIS systems and environmental and energy data to better understand urban activities. The 3D reality models generated with ContextCapture can be used for a variety of purposes, including urban planning, as illustrated by Helsinki 3D+ for the city of Helsinki, to detailed engineering design and construction workflows, as done by Huadong Engineering Corporation Limited for the City of Shenzhen. Digital cities enabled by ContextCapture are being created most effectively in China where the rate of change is dramatic and the need for better city planning is compelling. – Chintana Herrin, Product Marketing Manager, Reality Modeling, Bentley Systems.
Now that our Photos to 3D Model technology is released, we imagine global adoption for projects of all sizes. It works beautifully for historic buildings with crooked or sculpted façades. And it’s perfect for situations when a design has a texture of old brick or stone walls. The technique also works well for buildings that aren’t square, which prove more difficult to draw. Smaller firms will benefit from the cost-effective factor of the new technology, because they won’t have to blow their budgets on expensive resources. –Steve Johnson, vice president of product development at Vectorworks, Inc.
Reality Capture Third Party Products
Prior to CAD and BIM software companies taking on reality capture, products addressing reality capture were third party, who continue to develop their own state-of-the-art solutions to be used with building information modeling and other solutions.
The following are just a sampling of some recent technologies to hit the market that lend themselves to working in AEC:
Fyusion has created the product Fyuse version 2.0 that is akin to light field technology, which Google has been exploring for VR. The technology utilizes a number of cameras to capture rays of light moving through a prescribed volume of space, then merges them into one file. This if useful for someone with an Oculus type headset on so they can move their head around and get a photorealistic view without interruption.
Fyuse works like this: if you move your smartphone around an object the system is processing images simultaneously with data collected from other sensors in your phone to create a 3D model. So, it has more information than just that collected by the camera itself.
Matterport dropped the price on its Pro2 HD reality capture device with the announcement of the Pro2 Lite, that is now $2.5K instead of $4K.
What are the differences in these systems? The Pro2 Lite is physically lighter weight than the Pro2 HD with a weight of 6.8 lbs. as opposed to the Pro2 HD’s 7.6 lbs. The Pro2 Lite has a lower battery life of half the number of hours per charge (4 rather than 8).
With the Pro2 Lite you can scan 8,000 square feet at one time. Matterport offers fully immersive walkthroughs that they refer to as “True3D” and promises that transitions while moving will be smooth.
Indoor Reality, a company created by founder and CEO Avideh Zakhor, University of California at Berkeley, in 2018, announced the latest update to its 3D mobile mapping solution – multi-acquisition mapping – which can be used to rapidly capture a large area by using multiple devices in parallel, or the same device multiple times. As Zakhor explains, “Reality capture of large areas has always been a challenge for indoor mapping systems. Now AEC professionals can use the Indoor Reality handheld mapping device IR-500 to capture and process arbitrarily large areas in chunks, by allowing Indoor Reality’s proprietary software to stitch the acquisitions together seamlessly. All the auto-generated data products such as interactive web visualization, point clouds, 3D mesh and floor plan, are generated as if there was one gigantic acquisition, rather than many smaller ones. Even the tagging and annotations made on smaller acquisitions will be seamlessly merged into the combined larger capture.”
Not surprisingly, the Iranian-born Zakhor is no stranger to the tech industry, in fact, she is behind the technology that gave us Google Earth and StreetView.
What makes this particularly useful to BIM professionals is that the above capability coupled with Indoor Reality’s Revit Plug-in makes it possible to do rapid generation of BIM models for large areas at high speed.
AECCafe Voice asked Zakhor about Indoor Reality products and what makes them so successful. Please see the entire interview here.
Summary
Keeping up with this fast-paced industry is definitely challenging, but provides opportunity for organizations to offer low-cost, highly efficient solutions for adding value and accuracy to their design projects.
New third parties spring up constantly to provide responsive solutions for reality capture, refining and developing more efficiency and better integration of the process. Partnerships will continue to grow the technologies and bring a greater ease to including reality capture in upcoming 3D cities, telecommunications, transportation, real estate, urban planning, utilities, and many other complementary industries.
Autodesk