Sketch2Code uses AI to convert hand-written drawings to working HTML prototypes. Designers share ideas on a whiteboard, then changes are shown in the browser instantly.
The need
Creative design process begins with collaboration on a whiteboard where designers share ideas. Once a design is drawn, it is usually manually translated into a working HTML wireframe. This takes time and delays the design process.
The idea
We can use Computer Vision to build a system that understands what a designer has drawn on a whiteboard, then translates that understanding to HTML code. This way we can generate HTML code directly from a hand-drawn image.
The solution
Custom Vision service trains models to detect HTML objects, then uses text recognition to extract handwritten text in the design. By combining the object and the text, we can generate HTML snippets of different design elements.
