Researchers developed a system that uses neural cellular automata in 3D and with 50 kinds of blocks to grow single cubes into complex designs in Minecraft. Some examples include castles, trees, furnished apartment building, and even functional machines, like the caterpillar. And when they sliced a creation in half, it regenerated. Here is the summary of their paper recently published on arXiv:
Neural Cellular Automata (NCAs) have been proven effective in simulating morphogenetic processes, the continuous construction of complex structures from very few starting cells. Recent developments in NCAs lie in the 2D domain, namely reconstructing target images from a single pixel or infinitely growing 2D textures. In this work, we propose an extension of NCAs to 3D, utilizing 3D convolutions in the proposed neural network architecture. Minecraft is selected as the environment for our automaton since it allows the generation of both static structures and moving machines. We show that despite their simplicity, NCAs are capable of growing complex entities such as castles, apartment blocks, and trees, some of which are composed of over 3,000 blocks. Additionally, when trained for regeneration, the system is able to regrow parts of simple functional machines, significantly expanding the capabilities of simulated morphogenetic systems.
Credit: (footage) Shyam Sudhakaran/Djordje Grbic/Siyan Li/Adam Katona/Elias
Najarro/Claire Glanois/Sebastian Risi/ (music) bgmclub/Pond5