【技術解説】toio AR Math Game 『AR算数ゲーム』

Hi, I am an interaction engineer/designer MAO (@rainage) of The Designium.

In this article, I'm going to introduce the R&D experiment project - "toio AR Math Game".
This game is made to practice math for my daughter (first grade level of elementary school).

toio（トイオ）公式サイト

Kids could control the AR robot with a Mobile device to move the correct answer number, this makes arithmetic exercises more interesting.

The main technical core is "toio SDK for Unity" In the previous article (【技術解説】 漢字遊戯 x toio SDK for Unity), I have tried to use toio to read the standard ID for Unity.

[System Design]

In this work, I try to use more features of toio:
- Position ID: get the absolute position, angle of toio
- Move to point: automatically reach the specified coordinates
- Rotation: used to make more detailed robot responses

And, I chose AR Foundation to integrate the game interface. Use image track to set game orient anchor then update toio AR robot position by the position ID value.

And, I chose AR Foundation to integrate the game interface. Use image track to set game orient anchor then update toio AR robot position by the position ID value.

[Process & Issue]

(1). AR device controls the toio position:
We start to combine the toio position ID and AR Foundation for the 1st prototype, thanks to the basic AR prototype support by Matt(@mechpil0t). Set orient with print track marker on A4 (1/2 size of toio develop paper #02 ) and convert toio's 2D coordinate position into physical world position.

/* main target move script*/

foreach (var handle in cubeManager.syncHandles)
{
   Movement mv = handle.Move2Target(targetPos.x, targetPos.y, 25).Exec();
   if (mv.reached && started)
   {
       Debug.Log("target Reached.");
       started = false;
   }
}

(2). Merge AR objects on toio robot:
Then by obtaining the angle, I can overlap the AR image on toio. With this function, I can design more game features, ex. collision, particles...

/* read toio cube position & angle */
toUnityStr = string.Empty;
cubeManager.handles[0].Update();
toUnityStr = string.Format("pos({0},{1},{2})", cubeManager.handles[0].cube.pos.x, cubeManager.handles[0].cube.pos.y, cubeManager.handles[0].cube.angle);
readIDEvent.Invoke(toUnityStr);

(3). Position ID issue
After the success of the 1st prototype, I start to design the print layout for A3 (full size of toio develop paper #04) for a better user experience.

But at this step, I found the toio is out of control, sometimes it did not move to my point position (ex. (150, 750) → (150, 500) was worked, but (150, 500) → (150, 750) was not worked).

The reason is the toio default range setting (CenterX, CenterY, SizeX, SizeY, RangeX, RangeY) issue. If you change the Mat of toio SDK example, it only changes the display but does not change the cube move range. So we need to manual the range setting if used #04 as physical toio tracking paper. (the #02 is fine because the #02 range is in the default range)

So I updated my WebGL part to select which develop paper be used, then set the range depend which page is selected.

(4). More physical response
In addition to the images in AR, strengthen the connection between the game and the physical toio. I designed two behaviors to give feedback for physical toio. Then overlay these two behaviors with AR image real-time, This makes the robot more emotional.

[Correct] Turn around

[Wrong] Head shake

But at this step, I found another issue. If I do these behaviors while toio is stopped, it will be worked. But if I do these behaviors during toio is moving, it will fail. Because I could not interrupt the "Move2Target" method of SDK if the toio is still moving to the target position. I need to force to stop it at the current position then start to do the new behavior.

/* force the rotate behavior while doing move2target */
public void RotateCircle(int _duration = 570)
{
  StartCoroutine(CO_RotateCircle(_duration));
}
IEnumerator CO_RotateCircle(int _duration)
{
  // make target as current cube position
  started = false;
  Vector2 _pos = cubeManager.handles[0].cube.pos;
  cubeManager.handles[0].Move2Target(cubeManager.handles[0].cube.pos, 50).Exec();
  yield return new WaitUntil(() => Vector2.Distance(cubeManager.handles[0].cube.pos, _pos) < 10);
  // turn around
  cubeManager.handles[0].Move(0, 100, _duration);
}

[Conclusion]

The toio is a nice playful toolkit and the toio SDK for Unity is friendly for Unity developers. If you need to make some creative ideas for table scale position tracking or ID identify, toio is a good prototype tool for a stable position and fast responsive.

However, I think the toio SDK for Unity will be a bit challenging for unity beginners. Some additional debugging experience and device integration capabilities are required. The toio SDK document is very detailed, but the tutorial example parts are still a bit fragmented, and you need to compare the differences by yourself.

I really enjoy this development experience. It is very exciting that the physical device can be controlled through AR.

Editorial Note - 編集後記

こんにちは、広報のマリコ（@marikocco）です。今回は、SONYのロボットトイ”toio”を使った『AR算数ゲーム』の技術解説でした！Maoが6才の娘さんのために作ったというこちらのゲームは、今回ソニーさん主催で開催している『ロボットやろうぜ！」という作品動画コンテストにエントリーしています。

「ロボットやろうぜ！- toio & Unity 作品動画コンテスト -」開催決定 | toio blog | toio（トイオ）

子供向けのロボットトイですが、ロボットをつかったビジュアルプログラミングから本格的なプログラミングまで活用用途はとても幅広く、レゴブロックもくっつけたりできるのも魅力的なおもちゃです（もはやオモチャではないような）。ぜひ興味を持った方は、チェックしてみてくださいね✨コンテストで出てくる他の作品も楽しみです。

The Designium.inc
・オフィシャルサイト
・Interactive website
・遊んで学んで
・Twitter
・Facebook