3D Object Manipulation from a Single Photo

New software to manipulate objects in photographs using 3d models. The software can reveal hidden parts, maintain shadows, lighting, depth of field and fill resulting holes.

Incredible...
bremnetsays...

Nobody said easy, not sure it was suggested. They're showing the outcome, not the effort, which is truly amazing.

dooglesaid:

I call BS. That demo is heavily treated and worked on, not nearly as easy as they suggest.

bcglorfsays...

This is little, or almost no, different from integrating 3D animation into any movie. They've got some mechanisms to make it easier perhaps, but they don't really show how much easier they've made things.

billpayersays...

Did you even watch the indepth video ?

They've made it soooooo much easier.

Yes, Hollywood has been putting cg into footage for years but it require a team and tonnes of specialized software that cost thousands of dollars.
This is one app, with an immensely streamline workflow that most school kids could use.

bcglorfsaid:

This is little, or almost no, different from integrating 3D animation into any movie. They've got some mechanisms to make it easier perhaps, but they don't really show how much easier they've made things.

bcglorfsays...

I'm a Comp Sci grad who spent a great deal of time doing 3D coding so yes, I've got some idea what is involved here.

Best case scenario here is you have to track down an existing 3-D model that matches the object you want to manipulate close enough to do well. You also need that model's texturing to match close enough to look good. They don't clearly show how you map that model to a portion of your 2d image, but if they have made that relatively simple it is the 'big deal' portion they are showing off because that is very hard, and most likely has some finicky bits to it.

Also, the first bit of finding a good matching 3-d model is the killer. Armed with a well matched 3-D model, something like Blender already let you do this relatively easily. Finding that model is the hard part and for anything living it's simply not going to exist in 90% of cases, so your gonna just not do it, or do what the movie guys are already doing and build your own model.

I'm not saying there's not good work here, but I am sceptical of the fact that the real nuts and bolts of what would make this a 'big deal'(the UI mapping) isn't being shown. Furthermore, the animated origami clinches my skepticism. Sorry, but 3-D animation of 'some object' in your 2d image has NOT been made easy or IMHO been changed at all by their product. 100% of the effort there is the 3-D animation of the object, which you still have to get somebody to do artistically, full stop.

billpayersaid:

Did you even watch the indepth video ?

They've made it soooooo much easier.

Yes, Hollywood has been putting cg into footage for years but it require a team and tonnes of specialized software that cost thousands of dollars.
This is one app, with an immensely streamline workflow that most school kids could use.

billpayersays...

Dude, you still didn't watch the indepth video. Please do a tiny bit of research before you post.

They explain the model is stock. They explain how the app helps distort the model to fit the plate. They explain how they app figures out the texture blind spots. They explain how it mirrors existing textures to fill the spot. They explain how is figures out the perspective of the plate. They explain how it then matches the lighting and shading of the original.

bcglorfsaid:

I'm a Comp Sci grad who spent a great deal of time doing 3D coding so yes, I've got some idea what is involved here.

Best case scenario here is you have to track down an existing 3-D model that matches the object you want to manipulate close enough to do well. You also need that model's texturing to match close enough to look good. They don't clearly show how you map that model to a portion of your 2d image, but if they have made that relatively simple it is the 'big deal' portion they are showing off because that is very hard, and most likely has some finicky bits to it.

Also, the first bit of finding a good matching 3-d model is the killer. Armed with a well matched 3-D model, something like Blender already let you do this relatively easily. Finding that model is the hard part and for anything living it's simply not going to exist in 90% of cases, so your gonna just not do it, or do what the movie guys are already doing and build your own model.

I'm not saying there's not good work here, but I am sceptical of the fact that the real nuts and bolts of what would make this a 'big deal'(the UI mapping) isn't being shown. Furthermore, the animated origami clinches my skepticism. Sorry, but 3-D animation of 'some object' in your 2d image has NOT been made easy or IMHO been changed at all by their product. 100% of the effort there is the 3-D animation of the object, which you still have to get somebody to do artistically, full stop.

bcglorfsays...

Please give me a link to the 'in depth' video then. I've watched the 51 second one and the 5 minute one in your post. Model distortion and best guess texture filling are SEVERELY restricted by the quality of your model, which is just a rehash of what I said.

billpayersaid:

Dude, you still didn't watch the indepth video. Please do a tiny bit of research before you post.

They explain the model is stock. They explain how the app helps distort the model to fit the plate. They explain how they app figures out the texture blind spots. They explain how it mirrors existing textures to fill the spot. They explain how is figures out the perspective of the plate. They explain how it then matches the lighting and shading of the original.

billpayersays...

That was in depth enough for me to understand.
The model quality is completely open since you can use a near infinite resource know as the i.n.t.e.r.n.e.t. to get the model you need.

bcglorfsaid:

Please give me a link to the 'in depth' video then. I've watched the 51 second one and the 5 minute one in your post. Model distortion and best guess texture filling are SEVERELY restricted by the quality of your model, which is just a rehash of what I said.

bcglorfsays...

If only wishing or ignorance made reality go away. I suppose to manipulate that old picture of your parents you can just go to the magically internet repository with their high quality 3d models stored in it? Or you pet cat with the funny tail, or the tree in front of your house? Or the custom carved vase?

When you actually go try and match high quality 3d models to real world objects you quickly discover just how many are unique and hard to find.

billpayersaid:

That was in depth enough for me to understand.
The model quality is completely open since you can use a near infinite resource know as the i.n.t.e.r.n.e.t. to get the model you need.

billpayersays...

Nowhere do they claim it can do cats or your parents and only a joker would even write that.

Comp Sci Grad's seem to need everything spelled out for them. Go take a course on it.

Wasting no more time on this. bye bye.

bcglorfsaid:

If only wishing or ignorance made reality go away. I suppose to manipulate that old picture of your parents you can just go to the magically internet repository with their high quality 3d models stored in it? Or you pet cat with the funny tail, or the tree in front of your house? Or the custom carved vase?

When you actually go try and match high quality 3d models to real world objects you quickly discover just how many are unique and hard to find.

newtboysays...

...and apparently one can use a built in simple editor to modify the publicly available stock wire frame models to match them exactly to the item in the photo, movie, painting, etc. I see that it's likely more work that they imply, but far less work than previous editors or other ways of doing photo realistic CG. I give a tentative thumbs up.

billpayersaid:

The model quality is completely open since you can use a near infinite resource know as the i.n.t.e.r.n.e.t. to get the model you need.

newtboysays...

That's why they include a wire frame editor...so you can take a simple, basic model and modify it to match your individual cat nearly purrrrrfectly (if you spend enough time on the wire frame, and if the unseen parts of it are mirrors in texture and color to the known parts in the photo).
Really, I think that's not what this is for. It seems to be for modeling and manipulating every day objects in photo realistic ways with ease, not turning your parents around in the Christmas photos so they look like they're staring into each others eyes lovingly instead of both angrily turning their backs on the other! (although you COULD use it that way, if you're artistic and patient enough) ;-)

EDIT: One thing no one mentioned yet is it seems to also automatically edit IN the background, one less step to do manually.

bcglorfsaid:

If only wishing or ignorance made reality go away. I suppose to manipulate that old picture of your parents you can just go to the magically internet repository with their high quality 3d models stored in it? Or you pet cat with the funny tail, or the tree in front of your house? Or the custom carved vase?

When you actually go try and match high quality 3d models to real world objects you quickly discover just how many are unique and hard to find.

Send this Article to a Friend



Separate multiple emails with a comma (,); limit 5 recipients






Your email has been sent successfully!

Manage this Video in Your Playlists




notify when someone comments
X

This website uses cookies.

This website uses cookies to improve user experience. By using this website you consent to all cookies in accordance with our Privacy Policy.

I agree
  
Learn More