Iwata: You think about what may be possible years down the road and structure it accordingly.

Umezu: Yes. It doesn’t mean that we can keep it the way it is because what we released in the past is currently selling. Rather, it’s difficult because I have to purposely abandon the current situation and think of the next new thing.

Iwata: While the software developers are working as hard as they can on games for the hardware that’s now on sale, you are all alone thinking up the next hardware.

Umezu: I am! (laughs)

Iwata: Then you must have been thinking about what would come next about the time that the Nintendo DS system came out.

Umezu: Actually, when Nintendo DS came out, I had already finished SoC design, so I had begun thinking about what came after that!

Iwata: Oh, I see. Before even a single Nintendo DS system had been sold, you were already on to the next thing. What were you thinking about?

Umezu: First, I thought about the graphics—what kind of graphics would be required of the next generation system. Then I started fleshing it out.

Iwata: What kind of graphics did you think would be necessary in a future handheld gaming system at that time?

Umezu: I paid particular attention to home consoles. Even though it was a handheld, I wanted the visuals to be as close to those of a home console as possible.

Iwata: What did you focus on?

Umezu: I focused on the balance between graphics performance and power consumption. I couldn’t put in such a big battery that it would influence the size of the system, and handheld game systems are under particular constraints when it comes to battery life and cost, so first I thought about how to get the best performance within those limits. And unlike anything before, the Nintendo DS had the characteristic of two screens, so…

Iwata: You had begun work without yet knowing whether the world would accept Nintendo DS with its two screens.

Umezu: Right. So, at first, I also considered a design without two screens. That never saw the light of day, though.

Iwata: And you didn’t just consider it, you actually made it.

Umezu: Yes. I actually made something that ran. But while I was doing that, Nintendo DS became well received by a lot of people. About the time the Nintendo DSi5 system came out, I realized two screens was a must and began thinking about an early form of Nintendo 3DS.

Iwata: The Nintendo DSi system came out in November 2008, so…

Umezu: I began thinking about the SoC that would be an early incarnation of Nintendo 3DS in the latter half of 2007. I wanted to achieve high-performance graphics, but if I allotted all the power to graphics, I wouldn’t be able to do anything else later.

Iwata: At that time, we hadn’t started talking about an LCD that would allow users to see 3D with the naked eye.

Umezu: Right. You have to be thrifty when it comes to the limited power of a handheld, so I designed the SoC with some leeway for putting in whatever surprise elements might come along later.

Iwata: You had to design it with leeway so that you could adapt to whatever ideas popped up later on.

Umezu: Yes.

Iwata: The first thing (Nobuo) Nagai-san, who’s in charge at the Uji plant, said to me when he looked over the designs for Nintendo 3DS before mass production began was “This time it’s fully packed right from the start.”

Sugino: Oh, really? (laughs) But he was right. That’s why we don’t have any short-term plans for creating a more compact version of Nintendo 3DS like we did with Nintendo DS Lite. (laughs)

Iwata: Yeah. (laughs) When making Nintendo 3DS, we used all our tricks for Nintendo DS Lite—and more—for cutting down on size right from the start.

Konno: Yes. I think the timing was good. Some of the staff members around me were saying things like, “Today’s 3D LCDs really look good!” I thought so, too. I had a connection with 3D games anyway. After the development of Luigi’s Mansion for Nintendo GameCube was over, I was involved in the experiment of making a 3D version of it.

Iwata: Luigi’s Mansion 3D. Unfortunately, we never released it.

Konno: Yeah. We tried fitting the Nintendo GameCube with a small, roughly four-inch, LCD that allowed you to enjoy Luigi’s Mansion in glasses-free 3D.

Iwata: We showed that LCD as a reference exhibit at the 2002 E317, but kept the 3D aspect secret. I liked that, though.

Konno: Yeah. It had depth, so it really pulled you into the world of the game. I thought it was great, but…

Iwata: But we just couldn’t get past the problem of how to sell it.

Konno: Right. Liquid crystal was still expensive back then, and no matter how new an experience we could provide through the games, there would have been a need for players to buy the LCD as an accessory. There was even talk that it could turn out to be more expensive than the console itself! Iwata: In the end, we couldn’t overcome that hurdle and it never made it out into the light as a product.

Konno: Yes. I’d propose something, and they’d be like, “Huh? Are we still doing that?” (laughs) I decided to at least experiment. First we made it possible to represent Mario and Luigi in 3D. Then I talked to the development staff of Mario Kart Wii and asked if they could make it in 3D, and it went relatively well. A few weeks later, we could display Mario Kart Wii in 3D on the newest 3D panel. When I saw it, the 3D looked more natural than I had expected. I thought it was good and had some people at the company look at it.

Umezu: Yes. Making it 3D increased the power needed, and I ended up using all the extra power.

Iwata: Programmable shaders19 for showing shadows in 3D graphics have come into general use, but for Nintendo 3DS we adopted a technology from Digital Media Professionals (DMP) Could you explain that?

Umezu: What eats more electricity than anything in a handheld gaming device is the LCD backlight.

Iwata: Nintendo 3DS has two of those—and the upper one is for 3D.

Umezu: Right. And the 3D one has to delivery separate images to the left and right eyes. Which means that in 3D the amount of light delivered to each eye is halved. In order to make it look just as bright as usual, you have to increase the brightness of the backlight, which increases the power used by even more.

Iwata: Choosing 3D led to the need to double the graphics and increase the brightness of the backlight, so there must have been a lot of challenges with regard to power.

Umezu: I knew from the very start that the LCDs would require lots of power, so I thought we couldn’t use that much for graphics. I spent quite a lot of time looking into programmable shading, but when you use that method for a handheld game system, you also have software processing in the SoC, and a number of steps become necessary for calculating the shading to render one point on the screen, so it can’t keep up if the operating frequency isn’t fast enough.

Iwata: In other words, that method eats up too much power.

Umezu: Yes. We knew that DMP’s technology had the merit of calculating with the hardware, so we could cut down on the power used.

Iwata: DMP’s technology achieves with the hardware most of the shadow calculation usually done by a programmable shader, so even a slower operating frequency can achieve the same results. With today’s semiconductor technology, the frequency has a direct influence on power use, so it’s very beneficial in that regard.

Umezu: Yes. I thought DMP’s technology was best for saving power, given the limits of a handheld gaming system. What’s more, to save power just a little more, we prepared a power save mode for the backlight.

Iwata: Could you tell us what that does? Umezu: As mentioned earlier, what uses the most power in a handheld gaming device is the LCD backlight, and making one screen 3D presented the problem of even more power used by the backlight. The power save mode uses a technology called active backlight. It precisely controls the brightness of the backlight according to the brightness of the screen being displayed. When the screen as a whole is dark, the backlight itself gets darker, which saves power.

Iwata: In principle, it may not be much use in a game with lots of bright screens, but if you set the backlight to bright, the more darker screens there are in a game, the greater the effect of the power save mode.

Umezu: Right. But you can’t say exactly how much of an effect the power save mode is having on game processing because of the various factors that influence power use, such as how constantly the SoC is running, whether you’re using the camera or wireless, and how loud the volume is set.

Iwata: Maybe you can’t be exact, but how about an approximation?

Umezu: We announced the other day that the battery duration for playing Nintendo 3DS games on it is about three to five hours. When I measured it by playing several Nintendo games, with the backlight set to the brightest level and the power save mode turned off, battery duration was about three hours. But if you use the power save mode under the same conditions, it gets about 10-20% longer. And if you set the backlight to the darkest setting, the battery lasts five hours, but the power save mode makes less of a difference then.

Iwata: In other words, the brightness of the backlight has the greatest effect on battery duration.

Umezu: Right. What’s more, with the backlight set as high as it will go, battery life changes about 25% according to whether you’re playing in 2D or 3D.

Iwata: Yes, 3D is a battle against power use. How about wireless use? Some players may be worried about that.

Umezu: In StreetPass mode, the system isn’t always communicating, so there isn’t much of a drain on the battery, but games that communicate a lot through local play and online play—with the backlight set to high—have an over 10% effect on the battery.

Iwata: So it will be necessary for this system to be charged more often than for the previous systems. For that reason, we include a special charging cradle. We recommend that when players get home, they put their system there.

Another challenge we faced this time, after adopting the 3D LCD, is how it will look different to different people. The space between people’s eyes differs, among other individual differences.

Konno: Yes. We solved that with the 3D Depth Slider , which allows you to adjust the 3D effect.

Iwata: But, Sugino-san, as you were receiving all those requests to increase the number of switches, such as “we need to add 3D Depth Slider” and “there’s got to be a mechanical switch to shut off the wireless,” weren’t you and the other designers worried that this would turn out to be a product that felt messy if you could not place all these switchers properly.

Sugino: Yes, that’s right. For example, about the 3D Depth Slider, at first, we thought that if we were going to put that, a +/- switch would look smart and serve just fine.

Iwata: If you put in a physical mechanism for the slide, it takes up space. I suppose you wanted to avoid that since we were seeking a small, thin device.

Sugino: Yes. But when I asked if a +/- switch would be fine, they said it had to be a slide that could be freely adjusted however the user wanted. Miyamoto-san and Konno-san were especially particular about that. I said, “But isn’t the function the same?” Konno-san, however, made an actual prototype and showed it to me.

Konno: I wanted him to try it out, so in a relatively short period of time the tech staff put a slide on a Wii Remote. If you moved the slide halfway, the depth of 3D would change by half. I showed it to him and said, “See how great 3D is when you can adjust the degree of stereoscopy however you want?”

Iwata: Did it immediately win him over?

Konno: Yeah. (laughs)

Sugino: I thought it was definitely good. Iwata: With a slide, you know just where it’s at. It’s incredibly important to be able to go straight to where you want.

Sugino: That’s right. You can’t do that with a +/- switch. That analog aspect is great, so I knew we needed it.

In this project, Konno-san would come to me with a new suggestion and something that actually worked—like for the 3D screen and 3D Depth Slider—and say, “I made this. What do you think?” Actually trying something out, rather than just hearing an explanation, is much more convincing. Then I’d be like, “All right, let’s do it!”

Iwata: So for the 3D screen and other elements, you persuaded him by making a test model and having him try it out. (laughs)

Konno: Yeah. (laughs)

Iwata: Then, after you decided to put on the 3D Depth Slider, the idea came up of putting on two cameras. How did you decide on that?

Konno: A 3D camera was indispensable.

Iwata: It was just a matter of course?

Konno: Yeah. I thought, “If we can display in 3D, then we can add a 3D camera.”

Sugino: It was a fast decision.

Konno: Of course, the Nintendo DSi system has a camera, too, but taking 3D photos is a different kind of enjoyment, so I wanted to find some way to do it. Unlike the time we made Mario Kart Wii run in 3D, however, it took quite awhile before we could take stereoscopic photos and display them on the screen. We tried a number of approaches, and when we finally took something that looked 3D, I was really happy.

Iwata: What’s fun about 3D photos is how the people in them make all kinds of poses.

Konno: Yeah. When I turned the newly finished 3D camera on Miyamoto-san, he made a boxing pose and was like, “Whoa, it really sticks out!” (laughs)

Iwata: (laughs)


Sugino: We thought that no matter how hard we tried, this was the minimum.

Iwata: So you called it the “minimum model.”

Sugino: Later on, staff members told me that making this minimum model was a horrible thing to do (laughs)—that it was easy for the others to misunderstand that we would be able to make it that size.

Iwata: (laughs)

Konno: It is pretty small.

Sugino: When we made this model, it still didn’t have cameras.

Umezu: And it wasn’t 3D.

Konno: Right.

Sugino: Then we decided on the 3D LCD and the Circle Pad to make it more comfortable. Iwata-san, do you remember this prototype?

Iwata: Yes. You could change the location of the Circle Pad and +Control Pad like building blocks.

Sugino: We considered where to place the Circle Pad for the easiest play—over the + Control Pad or under it—by having people try out this model.

Konno: It was a working model, so we could actually play games.

Sugino: We even talked about releasing it just like that so players could put the buttons wherever they wanted. (laughs) We were joking, of course, but we went back and forth for quite a while with Konno-san’s team, saying, “It’s better up above,” and “No, down below.”

Iwata: There were scores of ideas about where the Circle Pad and +Control Pad should be, so it took awhile to decide.

Konno: Yes.

Sugino: Some even said that we should get rid of the +Control Pad and just have the Circle Pad. In the end, we put the Circle Pad up above.

Iwata: You tried out quite a few designs for the Circle Pad.

Sugino: Tons. Even after the 2010 E3, we kept adjusting the size and form of the Circle Pad.

Umezu: As mentioned earlier, I designed the chip without any thought of 3D, so we had to change the design quite a lot, like doubling the graphics rendering speed. I was really worried about whether we could make a chip that players could actually use in time for E3.

Iwata: And of course we couldn’t ask E3 to change its schedule on our account.

Umezu: Right. I asked those involved with the SoC to do the impossible, and with regard to polishing it off, I think Sugino-san had a rough time of it, too.

Iwata: I told him that I wanted several that looked like the final product for when I revealed the Nintendo 3DS system to everyone at the E3 media briefing.

Sugino: Yeah, that’s right.

Iwata: And I said that given the time constraints, I didn’t care if they used big ol’ things like sandwich boxes–as long as they ran, that is—for the ones that the players used in the exhibition hall. That’s how badly I wanted to exhibit working models at E3 so players could experience the 3D. But you actually prepared a lot that looked like the final product in the end.

Sugino: In the end, we prepared about 200.

Iwata: But when, at first, I said I just wanted a few, you said, “That’s absolutely impossible!”

Sugino: Yeah. I really thought it was impossible. But when I tried it, I did it! (laughs)

Sugino: I told the staff that you were fine with sandwich boxes, but they wouldn’t buy it, so they buckled down. When you think about the schedule after that, we needed to be bringing it to some semblance of form anyway. We needed to perform evaluative tests on a prototype before mass production.

Iwata: Like dropping it from a height of 1.5 meters, and heating it up, and cooling it down…

Sugino: And stepping on it.

Iwata: It only becomes fit for production after passing a variety of such tests and thereby gaining approval as a Nintendo product.

Sugino: Those tests were another reason I wanted to be in time for E3. But when we started making the prototype around January of 2010, the schedule was looking tight, so I couldn’t tell you we’d do it. Then, like a bolt out of the blue, the idea of putting in a gyro sensor came up.

Konno: Yeah, that’s right. (laughs)

Umezu: (holding his head) Oh, that…

Iwata: (laughs) The idea of 3D overturned a big tea table, but how about the gyro sensor?

Umezu: Hmm, it overturned a small tea table. To honest, I was a bit like, “Not again!” (laughs)

Everyone: (laughs)

Iwata: We decided early on to include a motion sensor in the Nintendo 3DS. Then we decided quite a bit later to add the gyro sensor.

Konno: About when was that?

Sugino: I think it was February or March of 2010.

Umezu: But earlier, I had said, “Nintendo 3DS specs are final!” (laughs)

Iwata: I do remember you saying, “Nothing else will fit in.” (laughs)

Sugino: That’s why I couldn’t guarantee that we would be in time for E3.

Umezu: Konno-san also demonstrated a gyro sensor to me.

Iwata: He hit you with another demo?! (laughs)

Konno: I thought the real thing would persuade him. (laughs) My department just happened to be working on putting the Wii console and the Wii MotionPlus as a set. I put the equipment on a cart and wheeled it into the meeting room and asked, “So how about this?”

Iwata: What kind of question is that?! (laughs)

Everyone: (laughs)

Iwata: The gyro sensor probably wouldn’t have come up if it weren’t for our experience with Wii MotionPlus.

Konno: I suppose not. I had heard Umezu-san say the specs were final, and I myself thought it might be difficult to achieve time-wise.

Iwata: But a certain someone wouldn’t let it go.

Konno: Right. (laughs) Miyamoto-san.

Iwata: When Miyamoto-san gets his heart set on something, he’ll say it’s all right to be late. I told him it wasn’t all right.

Umezu: I told him, too. “We’ll fall behind!” (laughs)

Konno: I thought, “Maybe it’s impossible this time?” and held my peace, but Miyamoto-san and those guys wheeled in a cart with the equipment and said things like, “Isn’t it fun this way?” and “See what a big difference it makes?” (laughs)

Iwata: The reason we decided to put in the gyro sensor was because he was just about as forceful as when we put a speaker in the Wii Remote controller just prior to mass production of the Wii system and the boat was about to sail. I thought it would come in handy somehow. I figured Miyamoto-san would do something later to prove that it was a good thing we had put it in.

Everyone: (laughs)

Iwata: Aside from Face Raiders, Nintendo 3DS has a lot of built-in software. You decided to put a lot of software into the system that wasn’t originally planned to be internal. Why did you decide to put so much into the system itself?

Konno: Maybe we overextended ourselves? (laughs)

Iwata: No, no! (laughs) Maybe it shows how much you love gadgets. Konno: Maybe so. For example, I really wanted to put in software like AR Games.

Iwata: We could have released AR Games as a separate, packaged game.

Konno: Yes, but we wanted it to be in the Nintendo 3DS from the start, so everyone who bought a Nintendo 3DS could experience that kind of 3D play.

Iwata: You had to walk around with it in Tag Mode, so if there weren’t lots of other people playing the same game at the same time, nothing much happened. When the Tag Mode in Dragon Quest IX: Sentinels of the Starry Skies became a social phenomenon, were you jealous?

Konno: Extremely. (laughs) When I found that a number of game data were exchanged with tag modes just by walking around with it, I thought, “Aw, man!” I have a child, and even though he hardly ever wants to go shopping with me, he wanted to use the Dragon Quest IX: Sentinels of the Starry Skies Tag Mode, so…

Full interview here