Monday, September 17, 2012

How to Decode QR Codes using Unity3D for Android (Zxing + Vuforia)

Following my previous blog post, here is my solution to decode QR codes on Android using Zxing and Vuforia.

UPDATE: If you also need to decode barcodes as well, here is a great solution. I haven't tried it though, I don't need to deal with barcodes as for now. My friend refers me to that page. I was late several months and didn't realize the solution was already out there :D oh well.

Step 1: on the blank Unity project, import Vuforia packages, and the Zxing.dll (provided at the end of this post, along with the camera prefab and this full script). Then add the ARCamera prefab to the scene, then add a script to it. Let's say it is CameraImageAccess.cs or whatever name you like.

Step 2: in the script, add
using com.google.zxing.qrcode;
then implement ITrackerEventHandler so that it will look like this
class CameraImageAccess : MonoBehaviour, ITrackerEventHandler
then define the implementation of  ITrackerEventHandler
 public void OnTrackablesUpdated () { try { if(!isFrameFormatSet) { isFrameFormatSet = CameraDevice.Instance.SetFrameFormat(Image.PIXEL_FORMAT.GRAYSCALE, true); } cameraFeed = CameraDevice.Instance.GetCameraImage(Image.PIXEL_FORMAT.GRAYSCALE); tempText = new QRCodeReader().decode(cameraFeed.Pixels, cameraFeed.BufferWidth, cameraFeed.BufferHeight).Text; } catch { // Fail to detect QR Code! } finally { if(!string.IsNullOrEmpty(tempText)) { qrText = tempText; } } }
OnTrackablesUpdated will be called by QCARBehaviour on every frame. So you could set a flag variable to determine whether you would need to decode QR or not, in this example we'll constantly try to decode QR.
I use Image.PIXEL_FORMAT.GRAYSCALE because we don't need colors to process QR codes.
tempText and qrText are just string variables to hold the decoded QR codes.
You could display qrText at OnGUI event or just use it as you need.

Finally, in the Start function, register this component to QCARBehaviour and initialize the camera
void Start () { QCARBehaviour qcarBehaviour = GetComponent(); if (qcarBehaviour) { qcarBehaviour.RegisterTrackerEventHandler(this); } isFrameFormatSet = CameraDevice.Instance.SetFrameFormat(Image.PIXEL_FORMAT.GRAYSCALE, true); InvokeRepeating("Autofocus", 1f, 2f); }
That's it, now we're done. Here is my Unity Package file containing the QRCamera prefab, the required scripts, and the Zxing.dll itself.
To use it, you just drag the QRCamera prefab to a blank/new scene. Then build it.

Final Notes:

  • This will only run on mobile devices (Android, and maybe IOS but I haven't test this on IOS, why? because I have no device/access to it, and I dislike Apple, so I won't ever buy Apple products). With that said, it will NOT run on Editor or any other platform, so don't forget to set the correct build target.
  • The zxing.dll used here is NOT the same dll file from my previous post. Now it receives byte[] parameter right from Vuforia as opposed to sbyte[] from Color32[] as in my previous post.
  • Lastly, the provided Unity Package here does not include all the Vuforia components, so you'll need to import them first yourself into your project.


25 comments:

  1. This comment has been removed by the author.

    ReplyDelete
  2. good work, but...when i tried to taget a qr code, it seems not be able to recogize it...
    I am using Galaxy s3

    ReplyDelete
    Replies
    1. Hi,
      Does the target QR code can be decoded successfully using other app (Barcode Scanner)?
      Do you have the QR code so I can test it on my device?

      Delete
    2. The QR code can be decode by other QR Code Scanner App like Qroid

      I generate an qr code from this web side http://goqr.me/
      type "this is a test"

      But ur QRCamera shows the camera view of QR Code, and keeps auto focusing, then no text shows on the bottom of window..

      Delete
    3. Hmm, I'm not sure what I'm missing here.
      I just tried it and it works fine, here's a screenshot:
      https://picasaweb.google.com/lh/photo/mdUmnsNALbftmEngppiJZRYTOQfO1PUb-Z_XMJNuF58?feat=directlink

      Do you have any error(s) in the logcat or something?

      Delete
  3. mmm..the log cat did not show any error
    what phone r u using?
    i saw ur pic looks like it should work fine
    Maybe i should try other phone..
    thanks

    ReplyDelete
    Replies
    1. My phone is Samsung Galaxy W, but I believe this should run well with other Android devices, especially your S3, it is far more advance.

      Delete
  4. there are warnning "no data sets defines.not loading any data sets"
    errors "faild to set frame format"...
    will that be problem?

    ReplyDelete
    Replies
    1. alright i found the problem.

      in the CameraImageAccess script, you must reinitialize camera frame format if it's failed.

      in my previous project i was lucky when i call CameraDevice.Instance.SetFrameFormat it runs immediately.

      in Unity when you run several scripts, the Start functions run in unknown order, perhaps the time of creations :-??

      so i suggest you create a flag variable to see whether SetFrameFormat is true or not, if not then try to SetFrameFormat before you call GetCameraImage (at OnTrackablesUpdated).

      that's why you got "failed to set frame format".
      while "no dataset" is no problem, it's because we don't use AR dataset.

      i will update the package later, it's sunday man, everybody must enjoy the day :D

      Delete
  5. It's wired...my cameraFeed is always null
    I think that's the problem...

    ReplyDelete
    Replies
    1. Hi, I just downloaded that Unity Package and create a new project using it, and it doesn't work :p
      I'm sorry, I'm investigating what script I missed to include :D
      Please wait...

      Delete
    2. //i've got it working now....^^
      //https://ar.qualcomm.at/content/getting-bytes-vuforia
      using System.Collections;

      using com.google.zxing.qrcode;
      using System;

      public class CameraImageAccess : MonoBehaviour, ITrackerEventHandler
      {
      private Image.PIXEL_FORMAT m_PixelFormat = Image.PIXEL_FORMAT.GRAYSCALE;
      private bool m_RegisteredFormat = false;
      //private bool m_LogInfo = true;
      private string m_QRCode;
      private string m_Info = "Scanning";
      private bool m_bGetQRCode = false;

      void Start ()
      {
      QCARBehaviour qcarBehaviour = (QCARBehaviour)FindObjectOfType (typeof(QCARBehaviour));
      if (qcarBehaviour)
      {
      qcarBehaviour.RegisterTrackerEventHandler (this);
      }

      InvokeRepeating("Autofocus", 1f, 2f);
      }

      void Autofocus () {
      CameraDevice.Instance.SetFocusMode(CameraDevice.FocusMode.FOCUS_MODE_TRIGGERAUTO);
      }

      public void OnTrackablesUpdated ()
      {
      if (!m_RegisteredFormat)
      {
      CameraDevice.Instance.SetFrameFormat (m_PixelFormat, true);
      m_RegisteredFormat = true;
      }

      if (!m_bGetQRCode)
      {
      CameraDevice cam = CameraDevice.Instance;
      Image image = cam.GetCameraImage (m_PixelFormat);

      if (image == null)
      {
      m_Info = m_PixelFormat + " image is not available yet";
      }
      else
      {
      string s = m_PixelFormat + " image: \n";
      s += " size: " + image.Width + "x" + image.Height + "\n";
      s += " bufferSize: " + image.BufferWidth + "x" + image.BufferHeight + "\n";
      s += " stride: " + image.Stride + " Pixel Count = " + image.Pixels.Length;
      //Debug.Log (s);
      m_Info = s;
      //m_LogInfo = false;
      string tempText = null;
      try {
      tempText = new QRCodeReader().decode(image.Pixels, image.BufferWidth, image.BufferHeight).Text;
      } catch (Exception ex) {
      m_Info = ex.Message;
      }
      finally
      {
      if (!string.IsNullOrEmpty(tempText))
      {
      m_bGetQRCode =true;
      m_QRCode = tempText;
      }
      }


      }
      }
      }

      void Update ()
      {
      if (Input.GetKeyDown(KeyCode.Escape))
      {
      Application.Quit();
      }
      }

      void OnGUI ()
      {
      //if (m_bGetQRCode)
      GUILayout.Label(m_Info);
      GUILayout.Box(m_QRCode);
      }
      }

      Delete
    3. Yes, that's what I said in my previous comment, SetFrameFormat before GetCameraImage at OnTrackablesUpdated.

      And, I got the original code from there too :D

      As I said, I was lucky with my previous project I just SetFrameFormat on Start and it just works, so I don't realize the problem. But thanks for bring this up.

      Delete
  6. Please can you paste hare link to your Zxing.dll edited library, because it isn't in package QRCam.unitypackage

    ReplyDelete
    Replies
    1. Ah ya sorry, my bad, I forgot to include it when reuploading that package.

      The link is updated now.

      Delete
  7. Hi, if I catch the QR CODE and then I load the another scene where I am using Vuforia for AR purpose and then i go back to QR catching scene, then i got always last cached QR Code.

    ReplyDelete
    Replies
    1. Hmm, do you by any chance store the QR text in a static variable?

      Delete
    2. I solve it. I must wait some time (1/4s) at OnTrackablesUpdated() before a get fresh data.

      Delete
  8. It is possible to use this scripts to a webplayer project?
    I've tryed just to change at the build settings and it didn't work.. what changes do I have to make?

    ReplyDelete
    Replies
    1. Hi, for non-Android project you could try this:
      http://ydaira.blogspot.com/2012/05/how-to-decode-qr-code-using-unity3d.html

      Because Vuforia plugin used in this page is for mobile devices.
      The solution on my other page doesn't use Vuforia. I haven't tried to publish that on web, you probably need to implement some mechanism to ask user permission to use their webcam.

      Delete
  9. Hi
    Firstly, brilliant effort in writing this project. But I have a few problems trying to implement it to a new unity project despite having installed Vuforia. Im getting a compiler message saying '"CameraImageAccess" does not implement interface member "ITrackerEventHandler.OnInitialized()"' How shall I fix this?
    Many thanks

    ReplyDelete
    Replies
    1. Hmm, just declare a function OnInitialized. I guess it's new in Vuforia, since I haven't update mine :p

      Delete
    2. Thanks,

      Also which camera shall I add the texture to? I tried using the AR camera from the Vuforia package but the texture is reflected on the y axis (mirrored) and wont decode :/ I may be very silly here...

      Delete
    3. Hi,

      I'm not sure about the mirrored Y axis, I haven't encountered that using Vuforia...

      For decoding QR just attach CameraImageAccess.cs to the camera, or alternatively just use the QRCamera prefab, and make sure there is only 1 camera in the scene.

      Hope that helps.

      Delete
  10. You can also download the commercial Qrcode read library for C# or VB directly, which can be easily added to your own engineering project.

    ReplyDelete