Supporting the RICOH360 business with technology
We asked Makoto Odamaki, the driving force and leader of the technology that supports RICOH360, about the technology and his own thoughts on 360-degree data.
―What type of functions were released with RICOH360?
■Using technology to support value unique to 360-degree images
I am working on image correction, AI auto cropping, and AI image enhancement functions that are built into “RICOH360 Tours”. These are provided as functions to make the work related to customer images significantly easier using machine learning technology. The image correction is used to automatically correct parameters of images taken by the customer such as brightness to more accurately communicate real estate property and products the customer wants to introduce.
The AI auto cropping is a technology that captures images automatically from a photo shot in 360 degrees. For work sites, it is necessary to shoot photos in a normal format, not just 360-degree photos, that are put on a website. At work sites, photos are taken after setting up the correct angle of view and other such factors, but the photographer may forget to shoot certain images, or the captured angle and orientation may not be quite sufficient. When this happens, the photographer must return to the work site to take more photos. With the AI Auto cropping, taking a photo of a property entirely in 360 degrees allows you to capture the appropriate images automatically from the spherical image and present them to the customer. There are also some customers who constantly deal with many properties from among these customers. I believe this function can significantly reduce the time it takes for the customer to shoot and edit images.
These AI functions are based on the millions of images of real estate property that we own. I believe it is because we have an extremely large volume of image data that we can implement functions applied with machine learning, enabling us to optimize the customer’s business.
Also, the AI-based image enhancement, a function that we are currently launching, is an initiative unique to Ricoh. This technology was jointly developed with an overseas research department and aims to provide a level of camera quality as seen with the high-end digital camera GR for users who are not quite satisfied with the image quality of 360-degree images using the conventional THETA models. By actually pairing and learning image data shot using GR with image data shot using THETA, we were able to create a system that can simultaneously enhance the resolution of 360-degree images and make image corrections such as reducing noise and correcting chromatic aberration. Honestly, we were unable to achieve the same level as GR, which was our initial goal, but we were able to significantly improve image quality. A paper written on this technology was accepted at CVPR, which is the premier conference on computer vision.(Click here for details
We have also developed various other technologies and also plan to provide many valuable functions in the future.
―How did you acquire the skills to work in this field?
■ Consistent involvement in image processing
I was involved in developing multi-function printers (MFP) as a hardware engineer when I first joined the company. Those were the early days of digital color copy machines, and I was able to experience a wide range of things from image processing algorithms to ASIC, electrical design, and system design. A great deal of image processing for MFP is now old-fashioned but after this I continued to expand my field of knowledge by such as working on camera-related image processing.
A significant turning point came when I studied overseas for two years as a visiting researcher in the Computer Vision Laboratory at Columbia University. I was originally involved in image processing but by honing my skills with the best in the world I was able to acquire the knowledge to enable me to drive image processing and computer vision forward at Ricoh. At the time, Professor Shree Nayar, who I am indebted to and is a famous researcher known to everyone in the field of computational photography, was already researching spherical cameras in the 1990s. His ideas and energy were astounding, and he was doing some incredible research on things like flexible lens cameras and cameras that operate forever by converting sunlight into energy while I was at the university. I was deeply impressed by his ability as a researcher to discover break-through technology for the world. Anyway, he would work on painstaking tasks, which suited me fine. My work during that period was published in “Communications of the ACM” as Cambits, a camera that can be assembled.(Click here for details
―What other type of work did you do at Ricoh?
■ Realizing the importance of business from experience of technology oriented themes
The most memorable theme was an IT machine project that connected a PC to an MFP. This theme was popular as a technological initiative at the time but the business scale could not be expanded so it was unfortunately shut down as a theme. I learned firsthand that it is extremely important to consider whether an initiative can be made into a business even if it is an excellent idea in a technological sense.
―What is required to connect technology to business?
■ Starting with research that finishes on a level that proves to be really useful to the user
I was studying for a while in the research department but felt strongly that I wanted to use my past experience to release technology to the market that would create value so I transferred to my current position in this group of the company.
Nowadays, machine learning or AI as it is known is booming, and with just a little study even a junior/senior high school student can handle AI. Various papers are released each day on the web, and you can quickly obtain code and run it even with surprisingly advanced processing. However, it is not enough to simply just use code available in the world to achieve the level where you can offer it as a product. Only the results that are convenient are reported in papers, and it is often the case that there are major drawbacks and side-effects. If you do not have the underlying camera and image processing knowledge to overcome these things, it will not be possible to finish the product on a level that satisfies the customer.
The team I am leading has members who have been making products with a particular passion for cameras and camera image quality for many years. They are able to finish with a product on a level that satisfies the customer using proprietary technology and knowledge to solve the next step of a problem where AI is not enough. This is simply where we are able to show our strength from working with cameras and image processing for many years in comparison to other companies who excel at AI.
■ Value of 360 degrees that increases
I think that 360-degree images have mainly two types of value. The first is being able to obtain various information including the structure by capturing everything around you. Everything is captured so you can cook it up however you like later on. I think the AI automatic capture is a good example of this.
The second is being able to obtain the customer’s interest and behavior based on where they are paying attention when the customer looks at a 360 image. This does not exist for other types of images. Isn’t this information the type of information that many people want to know about?
From these types of advantages of using 360-degree images, 360-degree related research is continuing to increase even in the field of research. As pioneers of the 360-degree camera, we also want to support this type of research, and we are developing various functions and technologies by ourselves. There are several core technologies within 360-degree image technology. I would like to continue releasing many functions to the market by building this group of technologies and increasing their width of application.
I believe we can dash to the top of the 360-degree world within one to two years, and while providing value to the market I would like to create a cycle in which we refine our skill and technologies.