Home > News content

Your graffiti is alive, CMU Chinese undergraduates make the paintings come true, there are codes and demos.

via:博客园     time:2019/6/16 14:32:39     readed:398

Guo Yizhen from the concave temple

Quotation Report Public Number QbitAI

In the world of mortals, painting is static. But in a mysterious computer world, everything is different, don't believe you.

Draw a unicorn and then gently.

The unicorn is "live" & rdquo;, running and jumping, leaping on paper.


Give it a name, and then a little —— the unicorn with the name, enters a three-dimensional world from the paper, begins to tirelessly climb, hills … …


In this world, there are still all kinds of strange creatures.

For example, snails that sang songs in the crowd.


Look, the monster with four feet:


The fisherman walking in the sky, afraid of being a fairy:


Ghostly gentleman with a hat


Even a grass and a tree can be refined:


Cartoon characters can also, you see there is a dream:


Yellow skin mouse Pikachu, this time no hair:


There is also a magical big face:


These little guys are all drawn by the people who eat melons.

In this virtual world, as long as a few graffiti, the painted things come true, can be called the real god pen Ma Liang. They ran in the same world, enchanting and entangled, like a ghost night.

This is the graffiti world created by the Chinese brother Lingdong Huang, who is studying at CMU. In this world, anyone can be like a pen, like Ma Liang, just take a picture and instantly become a reality.

Many people who have eaten melons have been surprised to see it:


This is really magic.


I really like these graffiti, I can watch it for a few days.


Just like poetry, they are all flesh-and-blood images, floating in a natural and casual place.

How did you do it

Obviously, it is a random painting, and it can automatically distinguish the "ghosts" of the little ghosts and "legs", and can also have rhythmic activities. How do these graffitis "live"?

“Creator” Huang took three steps to complete the construction of this world.

Add a skeleton to the graffiti

Just like the detection of key points in the human body, the graffiti drawn by hand also needs key points to determine the position of the bones in order to depict its essence.

Here, Huang used OpenCV's skeletonization tool and implemented it with Zhang-Suen's C++ code. However, this implementation traverses all the effects in the image, so the effect achieved on the web side is slower.

Huang had to re-use gpu.js to rewrite the code with the javascript branch of WebGL shaders.

It’s much faster after rewriting. For example, if we put a photo of a widowed sister with a gun, it will automatically focus on the picture, the widow's torso and the two arms holding the gun.


Let the machine understand the skeleton

However, now that the problem has come, seeing such a picture, humans can judge that it is the two arms of a person and TA, but the machine does not seem to work.

So, Huang decided to use the 8×8 window to scan the entire image, find the part of the mark with the stroke, as the root patch, and then look at the four sides of the root patch, which side is wired to which side. Extend, so that the entire skeleton can be found, and then the noise is reduced with an aggressive median-blur filter.


Now, the machine can finally understand the shape of the graffiti you draw, and the skeleton structure inside it.

Let the graffiti move

Now I have to find a way to make the graffiti move.

Here, Huang identified five major species: mammals, humans, birds, fish, and plants. According to the skeleton of the graffiti image, determine which species the small monsters are drawn into, and then assign body organs to different parts of the skeleton: this is the head. This is the arm, this is the leg ……

After that, he set up separate “walking” postures for different species. For example, plants should be jumping in a fixed place. Mammals and humans should be running on the ground. Birds should be placed in the sky to fan their wings. The fish should swim in the water.



With such a set of processes, the graffiti will move.

By the way, the world of graffiti monsters is “day, earth, water” and needs to be defined.

Huang used the 2D Gaussian function to multiply the Perlin noise to create a map similar to the island in the sea. The middle is high, the surrounding is low, and the outermost layer is water, so that the flowers and birds can get their own.

AI funny fans

Huang will graduate from CMU next year, after he was a Disney intern. This 97-year-old boy has created many magical projects.

For example, automatically generate landscape painting:


There is also the Emoji villain war:


Automatic Calligraphy Generator:


Quite interesting content, everyone can poke into the portal experience.

Professional & quot; Ma Liang & rdquo; company has been available

Graffiti moved, this function like the magic pen Ma Liang has actually been commercialized by a company.

Xiaoniu Technology, founded by Dr. Cao Xiang, a former Microsoft human-computer interaction expert, is a company that uses the self-developed Wonder Painter technology to turn various objects into moving roles.

For example, draw a girl in a dress in the painting interface:


Next, the girl can start the dance directly:


This technology of the Little Mavericks has been applied in some online marketing activities in the form of games. In the future, such technologies may also shine in the fields of art creation and children's education, which will greatly reduce the threshold of artistic creation.

Therefore, "God pen Ma Liang" is not only a cutting-edge technology, but also a looming future business prospect.


Demo experience:


I almost forgot to say that because this project is so popular, sometimes it needs to be queued like a drop to enter. The first Pikachu and Doraemon are in the second Demo server:


Original blog:


Skeletonization code:


China IT News APP

Download China IT News APP

Please rate this news

The average score will be displayed after you score.

Post comment

Do not see clearly? Click for a new code.

User comments