I worked on architecture layout optimization using Deep Reinforcement Learning as a Machine Learning Engineer at Spacewalk.

The company

Spacewalk is a Korean proptech start-up, located in Seoul.

They use Machine Learning and more precisely Reinforcement Learning to generate architecture designs at a reduced cost.

Projects

Path checking

My first task at the company was to write a module able to check the accessibility of parking spots (by car) inside a parcel.

Example of input :

Black : walls & core Green : roads Blue & cyan : parking spots (and spaces)

Initially, the project was supposed to be using RL (implemented algorithms for pathfinding were taking too long) in order to have results within 100ms per parcel.

<aside> 🎯 Skills

Python

Computational geometry

Shapely

Code optimization

Benchmarking

GEOS

</aside>

But I came up with a simple algorithm that could check the accessibility without the need for AI, just using computational geometry. Here are some examples :

Red : accessible / non accessible area

This algorithm is also much faster than the requirements : 1.14ms per parcel (±0.01ms), which is more than 87 times faster than the requirements !

Parking layout optimization

The next project I worked on is close to the first one : learn to optimize the layout of parking spots. For this, I used Deep Reinforcement Learning.

I implemented both the agent side (Pytorch, RLLib) and the environment side (OpenAI Gym, Shapely).

The environment implementation presented some challenges due to all the legal constraints that need to be respected when producing a parking layout.

I tried several algorithms (PPO, APEX) and architectures (simple MLP, Transformers, GNN).

After several iterations, I could successfully train a model reaching nearly optimal layouts (maximizing the first-floor area while placing the required number of parkings) :

<aside> 🎯 Skills

Python

Pytorch

RLLib

Deep RL

OpenAI Gym

Shapely

PPO

Transformers

Graph Networks

</aside>

This project was integrated into the main product (Landbook), and because sampling wasn’t needed at inference time (well-learned policy), the inference time greatly decreased.

This project was later used as the main reference for future works using DRL.

Floor plan generation

After my work on parkings and due to a shift in priorities, I then joined the main project, with a focus on improving the learning of the current agent.

My contributions are as follows :

• Use multiprocessing during training (reduce training time by a factor of 4)
• Use standardized data loaders (reduce training time by 20%)
• Implement a standardized interface for environment usage (OpenAI gym)
• Implement a standardized agent (StableBaselines3)
• Merge various repositories and refactor their code (roughly reducing maintenance costs by a factor of 3 and greatly improving collaboration within the ML team)

<aside> 🎯 Skills

Python

Pytorch

Deep RL

OpenAI Gym

StableBaselines3

Transformers

Multiprocessing

Refactoring

</aside>