Interview with TRI-AD: DevOps & Software Development in the Automotive Industry

Ayana Yokota Interview with TRI-AD

The future of software in the automotive industry requires an essential change in mindset for continued rapid development.

Hi, my name is Ayana Yokota and I’m a Developer Advocate at JFrog. I’m excited to introduce the first blog post in my JFrog customers interview series, highlighting unique use-cases and success stories.

This fascinating first interview was held with Mr. Yan and Mr. Moriya, from the Infrastructure Engineering team, and Mr. Etourneau from the Arene Tools team, in the Toyota Research Institute – Advanced Development, Inc. (TRI-AD).

  • Jack Yan: Senior Director
  • Satoru Moriya: Senior Engineer
  • Gwenn Etourneau: Senior Infrastructure Engineer

The automotive industry is currently at a turning point, which is something that only occurs once every 100 years. In this interview, they highlight the importance of not only making high quality vehicles, but also developing services that incorporate new technologies such as automated driving and connected cars.

Japan has been successful in the automotive industry on a global scale, and we are now entering an interesting phase in terms of how the next era will conclude. In this interview we had the opportunity to learn a lot about TRI-AD’s approach to software in the automotive industry.

TRI-AD Team

―Please tell us about your business, TRI-AD.

Yan: First, the name TRI-AD stands for “Toyota Research Institute – Advanced Development”. It was founded in March 2018 and aims to develop innovative software that recognizes “Mobility for All”, a vision of the Toyota Motor Corporation (Toyota). Our mission is to create new technology and advanced safety systems for the world.

We also recently directed Woven City, an incubator for designing smart cities that connect Toyota and its affiliates’ mobility and robotics technologies.

―It’s an exciting project that will support the daily life of the next generation.
Many people think of automobiles as hardware, but you mainly deal with software, right?

Yan: Well, TRI-AD is a software company. We develop software, for example, a mapping platform for automated driving. We don’t manufacture hardware, but in order to provide software, of course we have to think about and work with the hardware as well.

Only about 10% of the software we develop is installed in the car itself. The other 90% is software tools needed for developing software and distributing software to cars.

―I see. When you talk about automobiles, it sounds like there is a lot of specialized knowledge involved, but do most of your employees have experience in the automotive field?

Yan: A lot of people come from Toyota, Denso, and other related companies and have knowledge of the automotive industry. On the other hand, there are also people who come from other industries, such as finance and software development companies. We need to combine knowledge from both the automotive and software industries and make the most of it.

―So it’s an environment where people from different backgrounds can make a difference. I’d like to hear more about this later.
Can you tell us more about your work? What are the main responsibilities in your day-to-day work?

Yan: Our infrastructure engineering team manages the tools we need for development, JFrog Artifactory is one of them, and we also provide a VCS for managing the source code, for example. Our mission is to improve the productivity of the developers building our products by providing these tools.

We are also in charge of managing the cloud environment. We automate the management of cloud resources so that our developers can use the cloud every day without any inconvenience.

―What kind of people do you interact with in your work?

Etourneau: There are two main types of developers: those within TRI-AD and those at other companies, such as Toyota. We communicate with the developers involved on a daily basis. Right now, we do a lot of internal work, but eventually we would like to make our platform available to the public.

―So what are the difficulties in your work?

Yan: When you’re working with cars, you’re dealing with a lot of customized hardware, and one of the challenges is integrating development tools with them. Integration of those hardware and development tools is one of the challenges.

Etourneau: And it has to be done in a secure manner. Ultimately, you’re supporting the development of applications that run on top of the car, so you have to keep thinking about safety, and that’s also a challenge.

―What do you need to do to face the challenges? What are the useful tools that you have introduced?

Etourneau: We have some useful mechanisms in place, like the CI/CD pipeline, for example, but it’s not so much the tool itself that’s important, it’s the mindset, and in the case of CI/CD, we want to be able to track what happens when there’s a problem, so we’ll keep a log for some period of time as needed. The same thing can be said for automated driving simulations, being able to trace something back when there is an issue is important.

―Are there any other difficulties related to your work?

Yan: In terms of supporting internal developers, we also help teams to adopt DevOps smoothly, which can be a challenge.

Moriya: Traditionally, the automotive industry has done waterfall development, so we need to change our culture to one that incorporates DevOps. Some developers are having a hard time making that transition.

―Was it hard to move to change that kind of culture? Did you have any internal resistance or have to work hard to persuade people?

Moriya: I’ve never heard anyone say that the status quo is good, but even so, we have to change our way of thinking quite a bit, and that’s the hard part. Sometimes it’s difficult to change everything at once, so we’re working on it step by step.

Etourneau: Yeah. And I think it’s not just about embracing DevOps, I think it’s still about changing the mindset. As is often discussed in the context of agile development, there is a need to change the organization and the way projects are conducted.

I think this is a global trend that is not limited to teams. I think it would be great if we could adopt the same mindset not only when we are working on general software, but also when we are working on automated driving. It’s like Silicon Valley. That’s why we have training in agile development, and we share how to use the tools and how to integrate them.

―I understand that TRI-AD understands that this is an important aspect of the company and is working hard to make it happen.
Now that we’ve talked about agile, let’s dig deeper into that as well. Toyota Production System (TPS) is very famous in agile and lean development.

Yan: We’re embracing agile development now, but it’s very similar to TPS in that when a problem occurs, we stop the production line and solve it there. We also value the principle of Kaizen, which is the principle of continuous improvement in software development. We try to make software development work just like Toyota has used TPS to achieve success in the automotive manufacturing industry.

TRI-AD Team

―What specific methods or frameworks do you use in your development?

Yan: We’ve introduced Scrum framework and are working on turning in sprints every two weeks. We’ve been working remotely lately, so our Scrum events are done via video conference.

―So what skill set and mindset do you think developers should have in order to continue “agile Kaizen”?

Yan: In traditional software development, the goal was to execute a project that could last months or years and then release the final product. But to develop agilely, you need to be able to release in smaller chunks, quickly, and get feedback quickly. I think it’s important to have this mindset and to change towards it.

Etourneau: As Jack said, mindset is really important. If you’re used to working in a waterfall model, you basically work on one task, and then when you’re done, you move on to the next task, and so on. But in the agile world, you have to learn that you’ll also have to do multiple tasks simultaneously.

Releasing frequently in small increments shortens the feedback cycle from users and allows you to fix problems or change direction early on.

Supporting TRI-AD’s infrastructure by increasing the efficiency of in-house developers through continuous delivery and DevOps.

―Next, let’s hear about DevOps. As an introduction, can you tell us what you are working towards on a daily basis?

Etourneau: I think one of the goals of the team is to get developers in a position where they can spend more time on development. We don’t want our developers to spend their time setting up the tools they need or looking for good practices. We don’t want them to do their own provisioning, for example, or anything like that. In other words, we want to provide them with the tools and their best practices so that they have more time to focus on their development work and work more efficiently.

―You are the supporters of the developers who create your services. How often are you able to release those tools?

Etourneau: We have an automated pipeline of infrastructure tools that we can release within a few hours, and we have very little downtime. There is virtually no downtime. From the time we push the code to the development environment, to the staging environment, to the production environment, the release is virtually automatic. This is how we achieve continuous integration and continuous delivery, and I think that’s where our hard work comes in.

But product releases are also different, and it varies from team to team.

―It’s great to see that most of it is automated and even continuous delivery is achieved. Has the introduction of such a system and DevOps continued since the founding of the company?

Yan: Yeah, it was embraced from the beginning, and partly because our CEO, James Kuffner, is from Silicon Valley, but it’s a de facto standard for both Agile and DevOps.

―I’m happy to see JFrog Enterprise being used in such a forward-thinking environment. Thank you for choosing JFrog. Can you tell us why you chose JFrog?

Yan: We work closely with Toyota Research Institute (TRI), a company which is responsible for advanced research for automated driving in Silicon Valley. They are using Artifactory, so we adopted the same tool.

―Can you tell us about how you use JFrog? We’d love to hear about any advantages and disadvantages you have.

Moriya: I mainly use Artifactory to store the artifacts generated in the CI pipeline.

There are many strengths, but if I choose one, it is stable. One issue that comes to mind is the complexity of managing permissions from an infrastructure engineer’s perspective. You can set up detailed permissions, but I find the amount of permissions to be too much and difficult to manage. Also, Artifactory is a black box, so it’s difficult to troubleshoot it by ourselves.

―Thank you. It’s great to hear your honest opinion.

The Possibility and Excitement of Spreading the Results of “Kaizen”

―Now, finally, I want to ask you about the people and the team. What kind of people work at TRI-AD?

Yan: My team is a diverse group. For example, the member joining us today is Moriya-san, who was in charge of internal platforms and development tools at a major electronics company in Japan. Mr. Etourneau has experience driving infrastructure DevOps at a different Japanese company.

―It’s an environment where people from various backgrounds come together. What is the secret to getting these people to blend in right away after they join the company?

Yan: First, talking about the company as a whole, we have a standard post-employment program that is led by the HR Team. There is also a team called the Dojo, which implements internal training. We can take a variety of courses to improve our skills. I think the onboarding process is pretty comprehensive and good.

TRI-AD Team

―How should someone prepare before joining your company? Can you briefly tell us about the hiring process?

Yan: Because the official language of the company is English and there are many non-Japanese engineers to work with, I think having global work experience and way of thinking can be important. From a technical point of view, we look at knowledge and experience in infrastructure, automation, cloud, etc.

―It’s not easy to hire someone who meets all of these criteria.

Yan: You’re right, it’s not easy to find a technical, bilingual, cultural match. But the company and the team have a strong mission, and I think it is important that you take the mission in the same way we do. Fortunately, we have many applicants from all over the world.

―TRI-AD is working on really progressive projects, which I find very innovative. I’m excited about your automated driving and smart city concepts, which must be attracting a lot of people’s attention.

Incidentally, can you tell me why you joined TRI-AD?

Moriya: At my last job, I was doing research, so I was far removed from business. Now I’m much closer to it, and it’s very exciting to know that I might be able to apply my knowledge to have an impact on business, and I’m learning a lot.

―I see. I like the fact that your two careers are connected as well as your current one.

Do you find working at TRI-AD interesting?

Moriya: Yes, that’s right. What are the two main things that make your job worthwhile? The first is, as I mentioned, using the latest technology and applying it to the business. The other is changing the culture and mindset of our users. These are the most interesting and challenging parts.

―How about you, Yan? What makes it worthwhile?

Yan: I take pride in the fact that I’m working to improve the productivity of the developers closest to me and, in turn, Toyota’s development. Toyota is a very large company, so it’s rewarding to be there and continue to learn from a global perspective.

―Thank you very much for sharing your stories with us. I found your approach and thinking to be very interesting and a good example for Japanese engineers.
Lastly, could you give a message to organizations and engineers like TRI-AD that you would like to share with them?

Yan: It’s very difficult to embrace DevOps in an environment where you’re running legacy systems, isn’t it? There are two things I would say. First, rapid delivery and feedback requires the courage to embrace change and make the change. That’s why I think it’s important to be prepared to make the decisions you need to make for rapid release. The other thing you need to do is not be afraid to fail. You can’t achieve your goals without taking risks.

Moriya: What I can say is that “you can’t change in one day”. You have to work hard one step at a time.

This very interesting and passionate talk provided us insight into the TRI-AD culture and their DevOps practices. The fact that they were able to incorporate a new way of thinking into agile development and DevOps into their systems is a testament to their high level of technology. I was also impressed by the fact that they emphasized that to reach that point, you have to start by changing your mindset.

Hopefully this interview provided you with useful references and encouragement, please leave your comments on this page to let me know what you thought.