Experimental ARKit Calibration for Sewing Projectors

I've been experimenting with ARKit (The iOS API for building augmented reality apps) and I wanted to see if I could use it with my phone camera to calibrate my sewing projector. 

How it works:

• On the technical side, I'm using four unique markers of known size (ArUco markers) and ARKit. 
• The marker corners need to line up with the corners of the projection, for example, the bottom right corner of the bottom right marker lines up with the bottom right of the projection area.
• We move from corner to corner with the phone which automatically detects each marker, from here it will calculate the distance between each marker and do the calibration.

Challenges

• Getting consistent, accurate measurements. I'm still working on this as the measurements fluctuate. Sampling for at least 3 seconds at each marker has improved the accuracy. I found that capturing markers sequentially—moving from one marker to the next—yielded far more accurate results than trying to capture all markers at once.
• Designing a user experience that avoids showing the live camera feed—because that feed would end up projected, which really messes things up, but there also needs to be clear instructions on what to do. 
  

Results

The accuracy is not good enough at this stage, sometimes it gives me a near perfect calibration and other times it's off by several CM. 

I think something like this could be helpful for people who don't have a cutting mat or the cutting mat is not perfectly flat, but it need's to be consistently accurate. 

Check out the video below for a quick look at how it works: