måndag 12 augusti 2019

Stable version 1.4

A few weeks ago, Blender 2.80 was released, and thus it is finally time to release version 1.4 of the Daz Importer. It contains many improvements and bugfixes, but most importantly, it runs under Blender 2.80. It still also runs under Blender 2.79. Compared to version 1.3 there are also many improvements in Cycles materials, but most of these were already present in version 1.3.1.

Stable version 1.4 can be found at: https://www.dropbox.com/s/dvredkkiyyp1rp8/import-daz-v1.4-20190807.zip. It is identical to the development version as of today.

The documentation has also been updated to agree with this version: https://diffeomorphic.blogspot.com/p/daz-importer-version-14.html.

måndag 8 juli 2019

Add-ons

This post was updated on July 24, 2019 due to changes in the add-on interface.

Recently user engetudouiti wrote a script for setting roll angles. Although the script appears to be useful, I have not studied it in detail and don't have time to support it.  Instead, I thought it might be a good idea to create a mechanism that lets users add functionality to the Daz importer. In short, an add-on system.

Blender of course already has an add-on system, and you could write Blender add-ons that invoke the import-daz module, but that approach has some drawbacks. In particular, such "horizontal" calling will break if the name of the import-daz module is changed, e.g. to diffeomorphic-import-daz-bb15d1ca5708. A cleaner solution would be that add-ons for the Daz importer could be handled by the importer itself. The new add-on system in the development version does just that. This is work in progress and details will almost certainly change in the near future.

A Daz importer add-on is a python file that you put in the addons directory (under the import-daz directory, which itself is located under Blender's addon directory). It must contain the same information as a Blender add-on: a bl_info dict, and a register and an unregister function. Packages are supported; in that case the file __init__.py, must contain the bl_info dict and the register and unregister functions.

The file __init__.py in the addons folder should not be touched. It is an empty file which is necessary to make the addons directory into a Python package.

Information about the add-ons is found in the new Add-Ons menu. When no add-ons are loaded, it only contains the buttons Refresh and Save Settings.
When we press Refresh a list of add-ons appear below the buttons. All add-ons are loaded into memory, but they are not yet registered.
The interface is very similar to Blender's native add-on interface, cf https://docs.blender.org/manual/en/latest/preferences/addons.html. Each add-on has an arrow button, an enable checkbox, and the name of the add-on. The arrow button displays more information about the add-on, which is extracted from the bl_info dict.

When the checkbox is enabled, the add-on's register function is executed.
The Save Settings button saves a list of enabled add-ons. The next time Blender is started (or when the F8 key is pressed), these add-ons are loaded and enabled.









lördag 27 april 2019

Downsizing textures

DAZ characters usually have many large and beautiful textures. This can be a problem if you try to render a scene on an old computer with limited RAM memory, because once the memory is full Blender comes to a halt. However, for many purposes the texture resolution is unnecessarily high. Therefore, I have provided a utility program that downsizes all textures in a given directory, and a button in Blender that replaces the original textures with the downsized ones.

Install OpenCV

The script which resizes images cannot be used from inside Blender, because it uses the OpenCV library which is not included in Blender python. Installing OpenCV is quite complex, but the instructions at https://www.learnopencv.com/ are very useful.

Windows: https://www.learnopencv.com/install-opencv-4-on-windows/
Mac: https://www.learnopencv.com/install-opencv-4-on-macos/
Other operating systems: https://www.learnopencv.com/?s=install.

On Windows you start by installing the following software:
  1. Visual Studio.
  2. CMake.
  3. Anaconda. I installed the Python 3 version but Python 2 will probably work too.
  4. Git for Windows.
The instructions refer to install scripts whose location is not immediately obvious. The Windows versions can be found at https://github.com/spmallick/learnopencv/tree/master/InstallScripts/Windows-4.

Usage

In Blender, import a DAZ scene as usual. Save the blend file and press Save Local Textures.  This creates a subdirectory called textures and copies all used textures there. Note that this button has been moved from the Finishing section to the Materials section in the Setup panel.

The downsizing script is called resize.py and is located in the standalone folder. Once OpenCV has been installed, open the command prompt, go to the standalone folder, and type

activate OpenCV-master-py3
python resize.py <path to texture directory> <resize steps>


This creates new textures whose resolution is 1/2^steps of the original resolution, where steps is an integer between 0 and 4. E.g., if the original image has 4096x4096 pixels and steps = 2, the downsized image has 1024x1024 pixels. The total number of pixels is thus reduced by a factor 16.

In Blender, press Resize Textures, select the number of steps, and press OK. Clearly the resized images must exist in the right directory.

Results

Here are the results on my computer for a scene with a single nude Genesis 8 Female:



                   Step Total size  Render time
Original 0 24.9 Mb 59.24 s
25% 2 3.79 Mb 55.92 s
6.25% 4 342 kb 56.23 s

When going to maximal downsizing, the total disk space occupied by the textures is reduced by more than a factor 70. This is less than the factor 256 that one would naively expect for a bitmap, because the images are stored in compressed formats such as jpg or png. On the other hand, the render time (with Cycles on Blender 2.79b) does not change significantly in this case, because all textures fit into RAM memory. In realistic scenes, which contain multiple characters with hair and clothes, the original textures may fill up RAM memory and Blender starts swapping to disk. In that case the reduction in render time will be dramatic.

Update July 28, 2019:

When the Resize Textures button is pressed, a pop-up menu appears and asks for the number of rescale steps; one step means that the texture is rescaled by 50% in each direction, two steps that the scale factor is 25%, etc.

The name of the rescaled texture is the original filename followed by the scale factor multiplied by 100. So if the original filename is foo.png, the rescaled textures are named as follows:
StepsScale factorFile name
150%foo-050.png
225%foo-025.png
312.5%foo-012.png
46.25%foo-006.png

This button only replaces the textures in the Blender materials; the rescaled textures must already exist in the same directory. If you have access to some other program that resizes image files, you can of course use that instead of the standalone script bundled with the DAZ Importer, as long as you follow the naming convention above.

If you know that you will never need the original, high-resolution image files, you can alternatively overwrite the original filenames with the scaled files and reload the blend file.

söndag 31 mars 2019

AutoFace - How It Works

The core of AutoFace is the 3DMM face identity shape network described in

A. Tran, T. Hassner, I. Masi, G. Medioni, "Regressing Robust and Discriminative 3D Morphable Models with a very Deep Neural Network", in CVPR, 2017. The preprint version is available as arXiv:1612.04904 [cs.CV].

Code for this has generously been made available at Github, https://github.com/fengju514/Expression-Net.

The Expression-Net script analyzes a single input image and outputs a 3D shape in the form of weights for a set of 99 morphs labelled 00 to 98. The location of a mesh vertex is








The Expression-Net script also describes the 99 morphs, but only for a modified Basel Face Model (BFM), which appears to be popular among AI researchers. The BFM is a high-density face mesh, and as such rather unusable for artists, who prefer a medium-density full body mesh, which is rigged and textured. A key contribution of AutoFace is to translate the morphs from the modified BFM into shapekeys for two useful meshes, the Genesis 8 Male and Female characters used in DAZ Studio.

The pictures below show the first few morphs, for the modified BFM, Genesis 8 Male, and Genesis 8 Female. First the original meshes with no shapekeys applied.

Basic shapes - unmorphed meshes


Shapekey 00
Shapekey 01
Shapekey 02
Shapekey 03
Shapekey 04
Shapekey 05
Shapekey 06
A number of comments are in order.
  • The shapekeys in AutoFace are ten times stronger than the original ones, and hence the coefficients must be reduced by the same factor of ten. The reason is that higher shapekeys are quite subtle and difficult to detect.
  • The Genesis shapekeys taper off near the boundary of the BFM, in order to avoid a sharp transition to the rest of the head mesh. In particular a fattened jawline, such as the one in shapekey 00, is not transferred correctly. This problem may be affecting the example with president Trump.
  • They eyes, including the skin behind the eyes, are not morphed. Instead the eye vertices are scaled and translated based on the movement of the corners of the eyes. This ensures that the eyes remain round, which is desirable when posing.
  • Similarly, the inside of the mouth is scaled based on the movement of the corners of the mouth, in order to ensure that the jaws and teeth remain inside the face.
  • The shapekeys shown in this post are more or less symmetric. Higher shapekeys are less pronounced, and some of them are asymmetric.
As always, something is lost in translation, so it would be better if the deep network were trained directly for the Genesis 8 meshes. Or for subdivided versions of them. They high-frequency data could then be converted into a normal map. Alas, I neither have the competence, computer resources nor the training data to do this.



lördag 30 mars 2019

AutoFace - Create Rigged and Textured Character from a Single Photograph

During the last month I have been working on an old idea: a collection of scripts that creates a rigged and textured character from a single photograph. Today I am proud to announce the result: AutoFace. The scripts are usable, although not completely finished and the results are not perfect.

AutoFace documentation is found at http://diffeomorphic.blogspot.com/p/autoface.html.

A zip file can be downloaded from https://bitbucket.org/Diffeomorphic/autoface/downloads/.

Or if you prefer to clone the Mercurial repository: https://bitbucket.org/Diffeomorphic/autoface.

AutoFace uses quite a large number of external libraries, listed in the Prerequisites https://diffeomorphic.blogspot.com/p/prerequisites-1.html.


At the core of AutoFace is a deep convolutional neural network model and python code for robust estimation of the 3DMM face identity shape networks, directly from an unconstrained face image and without the use of face landmark detectors. The method is described in

A. Tran, T. Hassner, I. Masi, G. Medioni, "Regressing Robust and Discriminative 3D Morphable Models with a very Deep Neural Network", in CVPR, 2017. The preprint version is available as arXiv:1612.04904 [cs.CV].

Code from this project has been generously shared on Github: https://github.com/fengju514/Expression-Net. That site also contains code for estimating expressions and face poses, but AutoFace does not yet make use of this data.

Whereas the Expression-Net script does the job of recreating the 3D face from a photograph, the output is not really useful for the practising artist. The script outputs morphs for a modified Basel Face Model (BFM), which seems to be a favorite among AI researchers. The BFM is a high-density face mesh, but in practise we want a medium-density full body mesh, which is rigged and textured.

The goal of AutoFace is to transfer the output from the BFM to useful meshes, in particular the Genesis 8 Male and Female characters used in DAZ Studio. After the morphs have been transferred, we can load the characters into DAZ Studio and add hair, clothes and body morphs. The dressed characters can then be used in DAZ Studio or exported to other application. I personally use the DAZ importer to import the characters into Blender, where they can be posed and rendered.


The pictures below show some results for images of two beloved candidates of the 2016 US presidential election. This images are included in Autoface.

Input images, with landmarks detected by Dlib.
Basel Face Model
Genesis 8 Male and Female
Textures
Rendered
Clothes, hair and body morphs added in DAZ Studio.
Imported into Blender, posed and rendered.
As you can immediately tell from these images, the results are not perfect. The obvious flaw is that the characters look far too young. A possible workaround could be to add some aging morphs in DAZ Studio, although it is not satisfactory.

For best results we should start with a frontal photograph of a face in neutral position. The picture of secretary Clinton shows why. The deep network removes the smile from the 3D mesh, but it still lingers in the texture, in particular in the artifacts around the cheeks.






fredag 15 februari 2019

Loading morphs after anatomy has been merged.

You can load shapekeys to a mesh with the Import Morph(s) button. However, this only works as long as the mesh is intact, because morphs are stored as pairs of vertex numbers and offsets. If you change the mesh in any way, e.g. by merging anatomy or joining eyelashes to a Genesis 8 character, or if you delete or add any vertices manually, the vertex numbers are corrupted and the morphs can not be loaded. If you have spent a lot of time finishing your character, and then realize that you need to import another morph, what to do?

Fortunately, user engetudouiti came up with a clever trick for loading morphs to a merged character, using the plugin's ability to transfer shapekeys between meshes. This feature was originally intended for transferring corrective shapekeys from a character to her clothes, but here we use it backwards, and transfer shapekeys from a smaller mesh to the merged character.

Here is the recipe:

1. Import the character again on a new layer.
2. Delete all objects on this layer, except for those that you want to import morphs to.
3. Parent these meshes to the original armature. This step is necessary only if you use rig properties to drive the shapekeys, because the drivers look for the parents.
4. Import the morph to the new character or anatomy. This can now be done because it has the correct number of vertices.
5. Select the vertices of the merged mesh that are affected by the morphs.
6. With the merged character selected and the mesh with morphs active, press Transfer Other Shapekeys in the Morphs section.
7. Make sure that Selected Verts Only and Ignore Rigidity Groups are selected.
8. Delete the mesh with morphs.





The selection in step 5 is necessary because we transfer shapekeys from a small mesh to a bigger one, and we don't want vertices outside the small mesh's domain to be affected. Unfortunately, this step requires some manual work. This could have been avoided if you had created vertex groups containing the graft information before merging the meshes. If not, you must make the selection manually.

Here is how you would create the vertex groups:



With the character selected and the anatomy active, press Create Graft Groups in the Morphs section of the Advanced Setup panel. This creates two vertex groups for the character mesh: Graft_xxx and Mask_xxx, and one vertex group for the anatomy: Graft_xxx. Here xxx is the name of the anatomy mesh (not the object). After the anatomy has been merged with the character, the Graft_xxx vertex group consists of the vertices at the boundary of the anatomy. The vertices inside the boundary are those that should be affected by the morph and hence selected when the morphs are transferred.



tisdag 6 november 2018

Blender 2.80

The development version now runs on Blender 2.80. Or at least it runs on the version of Blender 2.80 that I downloaded a few days ago. There are still some issues, but most features seems to work as they do in Blender 2.79.

The most obvious difference is that the DAZ tab has moved from the T-shelf on the left of the viewport to the N-shelf on the right side. Not sure why add-ons cannot define tabs in the Tools shelf anymore, but so it is. Here is the same character imported into Blender 2.79 and Blender 2.80:

Blender 2.79


Blender 2.80
The development version can be downloaded from https://bitbucket.org/Diffeomorphic/import-daz.

The same changes have also been implemented on two of my other add-ons:
MakeWalk: https://bitbucket.org/Diffeomorphic/makewalk
MHX 2 importer: https://bitbucket.org/Diffeomorphic/mhx2-makehuman-exchange
Those add-ons are not really supported anymore, but at least the basic features now work under Blender 2.80.