Problem with resizing normal textures

Since Daz textures are often unnecessarily detailed and large, the Daz Importer includes a tool to lower the resolution and size of the textures, see http://diffeomorphic.blogspot.com/2019/10/resizing-textures.html. However, I recently noted a problem when it comes to resizing normal textures in TIF format: the resized normal textures react very differently with light than the original ones.

Here is an example with Victoria 8. The left image has the original textures (4096x4096 or 2048x2048). In the middle image, the diffuse, bump, specularity and translucency textures, which are JPEG or PNG files, have been downsized one step (to 2048x2048 or 1024x1024). In the right image, the normal maps, which are TIF files, have also been downsized one step (from 4096x4096 to 2048x2048). Clearly there is a problem at the texture seams after we downsized the normal maps.

It is not yet clear to me if this problem is specific to TIF files, or if similar problem would arise if the normal maps were stored in a different file format. However, in practice TIF files are rarely used except as normal textures. The latest commit has a workaround for this problem: we can now choose which file types are downsized.

Downsizing all file types except TIF (the defaults) avoids this problem, at least for many characters. And since all other textures are downsized, the memory requirements are still reduced significantly, although not quite as much as before.

Two new buttons (and a renamed one)

Eliminate Empties

Some assets create empties that are not really useful in Blender but mostly in the way. A typical example is this jacket with buttons, where each button consists of a mesh that is parented to an empty that is parented to a bone.

The new tool removes this extra layer of empties.

Select the parent armature and press Eliminate Empties. The empties are gone and the meshes are parented directly under the bone.

Eliminate Empties can be done before or after Merge Rigs. Or not at all if you wish to keep the empties.

Other vendors implement jacket buttons as empties that are instanced to a mesh. Instanced empties are important because they show up in renders, and therefore the Eliminate Empties button does not remove them.

Import Custom JCMs

The Import Correctives button imports Joint Corrective Morphs (JCM) for the active character. The Daz Importer has a build-in list of locations where it looks for such morphs, depending on the character type. This means that it will only find the standard JCMs provided by DAZ. However, some vendors provide custom JCMs for their  character. This is especially the case for monsters that deviate significantly from the standard character that they are based upon.

The new tool Import Custom JCMs opens a file selector which allows you to navigate to the location of the custom JCMs and load those that you need. In constrast, Import Standard JCMs, which is the new name of the old Load Correctives button, displays the list of standard corrective morphs provided by DAZ for the base character.

You can also load JCMs with Import Custom Morphs. That is not really useful, however, because then the shapekeys are driven by rig properties rather than bone rotations.

Transfer correctives should work for both standard and custom JCMs, since it transfers shapekeys driven by bone rotations irrespective of their origin.

Invoking the DAZ Importer from a script

The DAZ Importer is normally used from the UI panel, but it also possible to invoke it from a python script. To illustrate this, and to figure out how it could be done, I wrote a simple add-on which loads a duf file into Blender with specific settings, without the need for any user interaction. Add-ons for the DAZ Importer were described in an earlier blog post, but this is the first time that I actually used the add-on mechanism for something useful.

Press Refresh in the Add-Ons panel to read in a list of all available add-ons, and then enable the Sample addon for DAZ importer.

When the sample add-on is enabled, a new tab named Sample appears in the UI panel. It contains a single button named Import DAZ File, which invokes the DAZ Importer with specific arguments.

The add-on is located in the file sample-addon.py. Let us have a look at that file. The button is defined by the following code:

class SAMPLE_OT_ImportDazFile(bpy.types.Operator):
    bl_idname = "sample.import_daz_file"
    bl_label = "Import DAZ File"
    bl_description = "Import a specific duf file."
    bl_options = {'UNDO'}

    def execute(self, context):
        from ..error import getErrorMessage, setSilentMode
        # Turn off error popups
        setSilentMode(True)                
       
        bpy.ops.daz.import_daz(
            filepath = "/home/thomas/Dokument/DAZ 3D/Scenes/base8.duf",
            unitScale = 1/2.54,             # inches
            skinColor = (1, 1, 0, 1),       # yellow skin
            clothesColor = (0, 0, 1, 1),    # blue clothes
            brightenEyes = 1.5,             # brighter eyes
            fitMeshes = 'UNIQUE',           # Don't fit meshes.
            useAutoMaterials = False,       # Don't use auto shaders             handleOpaque = 'PRINCIPLED',    # Use principled node
            handleRefractive = 'PRINCIPLED',# Use principled node
            handleVolumetric = 'SSS',       # Subsurface scattering
            useEnvironment = False,         # Don't Load environment
            )

        print("Script finished")
        # The error message is the empty string if everything ok
        msg = getErrorMessage()
        print("Error message: \"%s\"" % msg)
        # Turn error popups on again 
        setSilentMode(False)                
        return {'FINISHED'}

The work is done by the operator bpy.ops.daz.import_daz, which invokes the DAZ Importer. The arguments are the same that appear to the right of the file selector when you press the Import DAZ File button.

This script loads a yellow Genesis 8 Female with blue basic wear and Tolouse hair, who is 70 inches = 5'10" tall.

If the operator encounters an error when executed the DAZ Importer displays a pop-up dialog. This is usually not desirable when it is invoked from a script, because the pop-up requires user interaction. Pop-up dialogs can be disabled by entering silent mode. In silent mode errors are only reported in the terminal window, but execution in not interrupted. The calls setSilentMode(True) and setSilentMode(False) enter and exit silent mode.

Silent mode can also be toggled on and off in the Settings panel, although it should always be off during interactive use.

You can check if the import operator encounters an error by retrieving the error message with getErrorMessage(). This function returns the empty string ("") if the operator succeeded completely, and a string starting with "ERROR" or "WARNING" otherwise.

Both setSilentMode and getErrorMessage are located in the file error.py, so you need to import them with the line

    from ..error import getErrorMessage, setSilentMode

Mini Material Editor Revisited

DAZ figures often have many materials that cover different parts of the body, like face, torso, arms, legs, etc. This makes it difficult to edit the materials in Blender, because we usually want to apply the same changes to all skin materials and it is difficult to miss something. For this reason the DAZ importer has the Mini Material Editor, which allows you to modify multiple materials simultaneously. The Mini Material Editor has just undergone a major overhaul to make it more intuitive and compresensive.

To start editing a selection of materials, press Launch Material Editor at the bottom of the Material section. The Material Editor appears.

The active object is a Genesis 8 Male. At the top we see the current material selection, which is set to Skin in this case. Below that follows a list of the materials which are affected by the edit.

The material selections are the same as in DAZ Studio. Then follows a button to Change Material Selection, and then the material settings, conveniently grouped.

Here we have opened up some groups of settings. To the left we see the original settings when the mesh has just been imported into Blender, and to the right we have made some modifications.

Here is the original node tree. Note how the values of the nodes correspond to the values to the left.

And here is the node tree with the changes applied. When a white color has been changed to something else, and there is a texture, a multiply node with the new color has been inserted. Similarly, if a texture is linked to a scalar socket with value 1.0, and the value is changed to something else, there is a new node which multiplies the texture with the appropriate value.

After pressing Reset Material, the node tree changes back. The extra Multiply nodes are still there, but they have no effect since they multiply a color with white.

The Change Material Selection button opens up an addition pop-up window, which allows you to choose a different material selection.

After changing the selection to Skin-Lips-Nails, we see that more materials will be affected by the editing.

If no material selection sets were specified in DAZ Studio for the active object, there are some default selections, based on the material properties. The default Skin and Skin-Lips-Nails selections are based on the material names, in the same way that the viewport color is.

In this case the material node tree is dominated by a principled node, presumably because the first material in the list is metallic.

Render Settings

Rendering in Blender does not only depend on the materials and lighting, but also on the render settings. This is especially important for DAZ characters, which may render completely wrong if the render settings are too conservative.

The picture above is a close-up of Olympia 8 with make-up, rendered in DAZ Studio. The eyebrows and the make-up are given in a separate texture, which is laid over the face texture (diffuse overlay). Moreover, the translucency color has been changed to green for clarity.

When we render the same character in Blender, the result may look like this, which is clearly unacceptable. However, this does not mean that there is something wrong with the materials. The image above was rendered with the Cycles integrator preset to Direct Light, i.e. with the settings for the Light Paths shown above.

If we change the settings as in the picture above, we get something which resembles the DAZ render much better. In order to see the green tint, the number of volume bounces is set to two. To make the eye transparent, the number of diffuse bounces is set to four, and reflective and refractive caustics are both turned on.

In the latest version of the DAZ Importer, the add-on automatically updates the render settings for Cycles if they are too low to render the scene correctly. You will then see the following warning message when the scene has been imported.

This behavior can be changed with the Render Settings option in the Materials section of the Settings panel.

 

• Ignore: Ignore insufficient render settings.
• Warn: Warn about insufficient render settings.
• Update: Update insufficient render settings (default).

New Export Script

An important part of the DAZ-to-Blender pipeline is exporting the final world-space coordinates of all vertices and bones. This step is necessary because it turned out to be too difficult to extract this information directly from the native DAZ files. Until now, the script that does this is called export_basic_data.dsa, and it exports a text file with the subscript .json.

The export script has now been replaced with the more appropriately named script export_to_blender.dsa. Like DAZ Studio itself, it compresses the exported file using gzip, which reduces the file size to one third. Compression has two advantages:
1. The expored files takes up less storage space, which is particularly important if you use an SSD disk.
2. They load faster, at least if they are stored on a mechanical hard disk, because less time is needed for disk access.
The new script exports files with the file extension .dbz (DAZ-Blender-Zipped). The Blender import add-on can read both .bdz files as well as the old .json file for backward compatibility.

To install the new export script, follow the same procedure as with the old script.

1. Choose Window > Workspace > Customize, or just hit F3.

2. Right click on Custom, and select Create New Custom Action.

3. Change both Menu Text and Description to Export To Blender.

4. Select export_to_blender.dsa in the to_daz_studio folder.

5. Drag the new action to the appropriate menu.

6. The export script appears in the File menu.

BVH Retargeter

The BVH Retargeter is an add-on for Blender 2.80 and later. Its purpose is loading animations from BVH files to a given armature, and editing these animations in various useful ways. It is an updated and improved version of the MakeWalk add-on which I developed as part of the MakeHuman team.

The BVH Retargeter can be downloaded from the following locations:

Stable version 2.0: https://www.dropbox.com/s/gmkwymwu7obrh8q/retarget-bvh-v2.0-20200329.zip

Git repository: https://bitbucket.org/Diffeomorphic/retarget-bvh/src/master/

Documentation: http://diffeomorphic.blogspot.com/p/bvh-retargeter.html

Retargeting a BVH animation to an existing character consists of three easy steps:

1. With the armature selected, press Load And Retarget.
2. Select the BVH file in the file selector.
3. The animation is loaded.

1. With the armature selected, press Load And Retarget.

2. Select the BVH file in the file selector.

3. The animation is loaded.