From Sketch to Space: Interactive 3D Environments from 2D Floor Plans

This project aims to construct a computational pipeline that converts 2D architectural floor plans into interactive 3D environments using Unity. The pipeline begins with raster image input, processed using OpenCV to recognize walls, doors, and windows. Extracted vectors are algorithmically converted into simplified 3D geometry, which is then imported into Unity's engine to enable spatial walkthroughs.

The objective is to enable designers and architecture students to quickly visualize their spatial ideas in a realistic setting through StableDiffusion, enhancing early-stage decision making.

Sample Sketch 1 - Contoured architecture drawing floor plan

Prompt: “Floor plan of a modern wooden ski resort cabin in Lake Tahoe”

Sample Sketch 2 - Rough abstract sketch with curve

Prompt: "Floor plan of a luxurious beach resort in Miami"

Sample Sketch 3 - Poche floor plan sketch straight

Prompt: "Isometric cutaway high-rise cyberpunk office based on the input floor-plan, glass curtain walls, neon-soaked night megacity skyline, rain-dappled windows reflecting magenta-cyan billboards, holographic desk HUDs, polished concrete puddles, hovering police spinner outside, volumetric blue fog, cinematic photorealism"

Sample Sketch 4 - Thin floor plan sketch straight

Prompt: "Concrete and aluminum factory based on floor plan, rustic vintage, industrial warehouse factory with windows and photorealistic lighting, brown drills and machines in the background"

Algorithmic Overview

  1. Canny Edge Detection


    Purpose: Detects edges based on image gradients using a multi-stage algorithm (smoothing, gradient calculation, non-maximum suppression, hysteresis thresholding).
    Strengths: Good at finding clean, sharp contours in raster input.

  2. Ramer–Douglas–Peucker (RDP) Algorithm


    Purpose: Simplifies polylines by recursively removing points that deviate the least from a straight path, preserving shape while reducing complexity.
    Strengths: Excellent for removing noise from complex contours.

  3. Corner Detection (Harris, Shi-Tomasi)


    Purpose: Identifies significant "turns" or junctions in geometry, helping to segment walls or capture key vertices.
    Strengths: Helps segment paths and distinguish straight vs. curved walls.
    Limitations: Often returns false positives in noisy drawings; pairing with line masks helps improve accuracy.