Scripting for version 1.6

It was more than a year since I last wrote about scripting the DAZ Importer, and some things have changed since then. In particular, the scripting interface now uses snake case like the official Blender API. In contrast, the DAZ Importer source code uses camel case. This was an unfortunate choice but it is too late to change that.

In particular, this means that

import_daz.setSelection(list_of_filepaths)

is replaced by

import_daz.set_selection(list_of_filepaths)

Apart from that change, most of the examples in the previous post should work without changes. A description of the available functions and operators are found in

• DAZ Importer Functions
• DAZ Importer Operators 

All functions intended for external scripting are defined in the file api.py, so you could also look in that file.

I added a new repository with some sample scripts. It has a dual purpose: to provide some examples how you can script the DAZ Importer, and also to act as a test suite for future releases. The sample scripts are found in

https://bitbucket.org/Diffeomorphic/daz-importer-scripts/

 

Morphing Armatures

Update:

Surody discovered that automatic armature morphing does not work when rendering animations. To circumvent this problem, a new tool for rendering a sequence of frames has been introduced, see Morphing Armatures and Rendering.

Original post:

In DAZ Studio it is possible to create morphs that change both the character mesh and armature, so called ERC (Enhanced Remote Control) morphs. In Blender that would correspond to morphs that change the rest pose of an armature, but that is not supported by Blender. In the past such ERC morphs have been ignored, but with the latest commit of the DAZ Importer they can be dealt with.

In order for ERC morphs to be loaded, the global setting ERC Morphs option must be enabled, otherwise morphing armatures will not work. In most cases it is not necessary to morph armatures, and by disabling this option we can eliminate some overhead.

Assume that we want to animate how dr Jekyll is transformed into mr Hyde. Here we have three frames from that sequence, with the Hyde morph at 0%, 50%, and 100% respectively. The mesh is transformed nicely, but the armature does not follow suit.

There is a new tool that takes care of that. After the value for the Hyde morph has been set, press the Morph Armature button in the Morphs panel.

The armature now jumps into place. Here is mr Hyde at 50% and 100% with his morphed armature. The bone lengths do not change even though the distance between the bones has increased, but that does not really matter. What matters is that the bone heads, i.e. the pivot points, are at the right location. We see that the lCarpal4 bone does not move for some reason, probably because it has been misspelled.

That the pivot points are located correctly is important when the character is posed. Observe the right shoulder in the picture above.

It is inconvenient to have to press the Morph Armature button for each frame change, and you may wonder if there is a way to automate that. The answer is yes, but the procedure is somewhat awkward. However, it is awkward on purpose, for two reasons.

• Morphing armatures is a quite expensive operation, which we don't want to enable by accident. E.g., the frame rate for the Jekyll-to-Hyde sequence dropped from 54 fps to 8 fps when auto-morphing armatures was enabled.
• Auto-morphing relies on application handlers, which would not be defined if the DAZ Importer is disabled, or if the blend file is moved to another computer, e.g. at a render farm.

To make auto-morphing work even when the DAZ Importer is disabled, we have to load the file morph_armature.py into the text editor and register it, in the same way as the stripped runtime system was loaded in previous releases of the DAZ Importer.

To enable auto-morphing in a single session:

1. Open the file morph_armature.py in a text editor window. It is located in the runtime subfolder.
2. Run the script by pressing the right-pointing triangle button (Run Script).

To make it persistent, so auto-morphing is enabled automatically every time the blend file is opened. This is what you want to do if you send the file to a render farm.

1. Open the file morph_armature.py.
2. Enable the Text > Register checkbox.
3. Save the blend file.
4. Reopen the blend file.
   

When automorphing is enabled, only selected armatures are morphed. This can be used to control which armatures are morphed, although deselecting an armature does not change performance very much. The handler toggles into edit mode once for each frame change and updates the rest pose of all selected armatures. Typically going in and out of edit mode is the expensive operation, and that is only done once independent on the number of armatures.

Here we see the difference between the armature being selected or not.

DAZ Importer 1.6.0 Released

 This release consists of four separate parts:

• DAZ Importer, a Blender add-on for importing native DAZ Studio files (*.duf, *.dsf) into Blender. It also contains some tools to make the assets more animation friendly. 
• MHX Runtime System, a Blender add-on for posing the MHX rig.
  
• DAZ HD Morphs, a Blender add-on by Xin for importing HD meshes and morphs into Blender.
• Diffeo HD Exporter, a DAZ Studio plugin by Donald Dade to export HD scenes to Blender.

 

Download version 1.6.0 from:

https://www.dropbox.com/s/abda1ext7wbst97/import_daz_mhx_rts_v1_6_0.zip

 

Main documentation page: 

https://diffeomorphic.blogspot.com/p/daz-importer-version-16.html

 

This release is compatible with Blender 2.80 and later. The last version compatible with Blender 2.79 was version 1.5.1.

 

Major new features in this release:

• New morphing system using intermediate properties which correctly reproduce morphs in DAZ Studio. How to implement this was explained to me by Engetudouiti.
• New morphing system using simple expressions instead of slow python scripts, as suggested by Xin. This improves performance, in particular on GPUs. It also makes it possible to use expressions on systems where the DAZ Importer is not enabled, e.g. render farms, making the stripped runtime system of previous releases obsolete.
• Easy Import makes it possible to import a character and make it animation-ready in a single step, as suggested by Alessandro Padovani.
  
• Correct FK-IK snapping for the MHX rig, following suggestions by Engetudouiti.
• Tools for dealing with MHX animations, not just single poses.
• Improved user interface for the MHX rig.
  
• A tool for converting HD morphs to normal or displacement maps (the DAZ HD Morphs add-on by Xin), and tools for importing these maps into Blender and drive them with the morph strength.
• A tool for fast export of HD meshes from DAZ Studio (Diffeo HD Exporter by Donald Dade).
• Basic support for simulations.
  
• Many bug-fixes and minor improvements.

This release will probably be the final one when it comes to new features. Version 1.0 of the DAZ Importer was released in 2016, but I looked into importing native DAZ files as early as 2011, so the add-on is already ten years old. It has grown into a much more ambitious project than I ever expected, and is quite enough for my own needs. I will continue to fix bugs and adapt the code to new releases of Blender and DAZ Studio, though.

ERC Morphs and FACS

Enhanced Remote Control (ERC) consists of a number of methods for remotely controlling various channels in DAZ Studio. It can be viewed as generalized morphs, which in practice come in three types:

1. Shapekeys that change the mesh.
2. Bone translations, rotations and scale transformations that affect the meshes.
   
3. Changes to the bones' pivot point without affecting the meshes.

The first type corresponds to driven shapekeys in Blender, and the second to drivers for the bones' location, rotation and scale channels. However, the third type has no counterpart in Blender. It would correspond to morphs that modify the head and tail of bones in edit mode, but in Blender this is impossible. 

Old-fashioned morphs for Genesis 3 and 8 consist mainly of bone poses of the second type, accompanied by additional shapekeys, and they work well in Blender. (Even more old-fashioned morphs for Genesis 1 and 2 are pure shapekeys.) However, the FACS morphs for Genesis 8.1 mostly consist of shapekeys together with changes in the pivot points, i.e. a combination of type 1 and 3, although some morphs rotate bones where rotation is necessary (eyes, eyelids, jaw and tongue). This can lead to problems in some cases, in particular with tongue morphs.

To the left we have the FACS Tongue Out morph. A shapekey moves the tongue mesh out of the mouth, but the tongue bones do not follow. In DAZ Studio the bones are also moved without affecting the mesh further. That would correspond to the change in edit mode to the right. However, these changes are ignored by the plugin, since it is impossible to drive the rest pose location in Blender.

 

This problem does not affect the mesh as long as we only combine this morph with other morphs that also work with shapekeys. However, if we combine it with morphs that rotate the bones, we may have problems.

Here we have such a case, where Tongue Out is combined with Tongue Right and Tongue Tip Bend. The two latter morphs rotate the tongue, and their influence it too big because the pivot points remain too far back. Note that the FACS morphs have non-zero final values, to the right, even though the sliders are zero. This is because another morph drives the FACS morphs.

We can compensate by increasing the Tongue Tip Bend slider. That increases the final value of this morph (i.e. it decreases the magnitude of the final value), and the tongue no longer intersects the skin.

Adding morphs after lashes and geografts have been merged

Update August 13: 

Unfortunately, this feature causes performance problems in the viewport. It can therefore be turned off. Once you have finished building your character, you can press Finalize Meshes in the Finishing section to improve performance. This permanently removes the possibility to add new morphs, and some other build-time features as well, so make sure to save a copy of your blend file before doing this.

Original post:

These days I mostly use Easy Import to get my DAZ characters into Blender, with a list of favorite morphs. Alas, it quite often happens that I find that there is some additional morphs that I want to load. Since eyelashes and often geografts have already been merged with the main character mesh, this has not been possible because the mesh topology has been changed. Until now.

In the development version it is now possible to add new morphs, even if the character mesh has been changed in certain ways. More precisely, it works if

• Eyelashes (or other meshes) have been added to the mesh with the join command.
• Geografts have been merged with the Merge Geografts tool.
• Vertices have been added to the mesh in edit mode.

It does not work if some vertices have been removed from the mesh in edit mode. Removing vertices changes vertex numbers which are used when loading morphs. The Merge Geograft button removes vertices too, but in this case the mesh keeps a list of associations between current and original vertices.

Morphs are only loaded to the original vertices, or what remains of them. This means that it is not meaningful to load FBMs to a mesh with merged geografts, or FHMs to a mesh with merged lashes, because the shapekeys cannot be transferred. But FBHs and FHMs usually involve changes to the armature rest pose which cannot be loaded anyway.

This is an advanced feature since it is quite easy to import faulty shapekeys if the mesh has been accidentally changed, and therefore it is turned off by default. To turn it on, enable the Load To Modified Meshes global setting.

I have been using this feature for a couple of days and find it very useful, although I am not sure about best practice yet. Perhaps one should keep eyelashes as a separate mesh, in case one wants to transfer shapekeys to them later.

Changing Outfits

Update 2021-10-03

The name of the retargeting tool was changed in version 1.6.0 to Change Armature. It is now located in the Setup > Corrections panel.

Original post

Engetudouiti requested a tool for changing armature modifiers, in order to change clothes of a given character. Using different outfits for the same character is something that I have thought about myself. There are several ways that can be done, but all have their own disadvantages.

One possibility is to create a super-outfit that contains all outfits that the character will ever need, and then put the indidual outfits in subcollections that can be excluded from the scene when not used. This makes the file very heavy, and it takes a long time to load it. It also requires exceptionally good planning. If we later realize that we want to add another outfit to the character, the entire super-file must be reloaded.

Another possibility is to create separate files in DAZ Studio, one for each outfit, starting from the same nude character. This is the method that I have used a lot recently, but it also means work duplication. For each outfit, all morphs must be reloaded. This may not involve much manual work if you use easy import and favorite morphs, but if you then tweak the character in Blender, those tweaks must be repeated for each outfit.

In this post I will describe a third method, which is both flexible and avoids work duplication. The idea is to import the nude character only once, and then import each outfit from a separate file and merge the outfit with the nude character.

DAZ Studio

In DAZ Studio we will create the following files:

1. The shape file which only contains the character itself. Not even the eyelashes need to be included in this file.
   
2. The nude file which contains everything that makes up the nude character: body, eyelashes, hair, geografts, and geometry shells.
3. One file for each outfit.
   

Create the character shape. Save the file as aiko-shape.duf. We do not need to export this file to Blender.

Complete the nude character by adding hair, geografts, and geometry shells. Save the file as aiko-nude.duf and export the file to Blender (aiko-nude.dbz).

If we have several shells it may be necessary to change the visibility of some surfaces.

Open the file aiko-shape.duf again and add the first outfit. Save the file as aiko-basic-wear.duf and export to Blender.

Open the file aiko-shape.duf again and add the second outfit. Save the file as aiko-bardot.duf and export to Blender. Your directory should contain the following files:

We are now done in DAZ Studio and switch to Blender.

Import the nude character

We will use Easy Import to import the duf files into Blender. Below the Easy Import DAZ button we have a field where the path to the favorite morphs can be specified. If we press on the folder icon to the right, a file selector opens where we can choose the path to the file with favorites.This button is located in the panel because a file selector can not be lauched from inside another file selector.

Select the aiko-nude.duf file and make the following settings:

• Merge Rigs: We need to merge the eyelash, hair and geograft rigs with the main rig.
  
• Use Favorite Morphs: The path to the favorite morphs that we specified before appears here too. However, there is no folder icon since we cannot lauch a file selector from here.
• Transfer Shapekeys: Disabled, because there are no clothes that we need to transfer shapekeys to. If enabled, the tool will try to transfer shapekeys to the hair which normally unnecessary.
  
• Merge Geografts: Shapekeys whose body part setting is Body or All are transferred to geografts before merging, even though Transfer Shapekeys is off.
  
• Merge Lashes: Shapekeys whose body part setting is Face or All are transferred to eyelashes before merging, even though Transfer Shapekeys is off.
• Rig Type: Set to DAZ.  This is necessary because the Copy Pose tool that we will use later does not work with the MHX or Rigify rigs. Rig conversion can be done manually in the last step.
  

After some time (a few minutes) the nude character appears in Blender. Save the file as aiko-nude.blend.

Import the first outfit

Open the nude file again and save it as aiko-basic-wear.blend. Open the Easy Import tool and select the aiko-basic-wear.duf file. This time we set the clothes color to  blue and only enable the first three options.

Now we have two collections: the nude collection and the basic wear collection that we just imported. The character mesh in the second collection has now served its purpose and can be deleted.

Select the clothes and press Retarget Armature Modifiers
 

In the popup dialog, select the rig in the nude collection, which Blender has renamed to Aiko 8.002. This changes the targets of all armature modifiers, and also the parent object. Move the collection with the meshes to the Aiko-nude collection, and give it a better name. The remaining objects in the Aiko-basic-wear collection are not needed anymore and can be deleted.

We can pose the character to confirm that the clothes is controlled by the same armature as the body and hair. One final step is needed before we are done: shapekeys must be transferred to the clothes.

With the clothes selected and the body mesh active, press Transfer Shapekeys in the Morphs section.

Since the clothes only cover the body, we set the Body part field to Body. This singles out the JCM morphs which what we want to transfer.

And now the skin is not poking through when the character is posed.

Import the second outfit

The Bardot outfit will pose some new problems.

 

 Open the file aiko-nude.blend again and save it as aiko-bardot.blend. In Easy Import we again leave the three first options enabled and import the file aiko-bardot.duf, and then remove the extra body mesh that we don't need.

The first issue is that the feet and the sandals have different rest poses.

With the body armature selected and the clothes armature active, press Copy Pose. This will pose the body armature so it fits the shoes. Unfortunately, the Copy Pose tool only works with straightforward rigs such as the native Genesis rigs, but it does not work with MHX and Rigify. This is the reason why we set the rig type to DAZ in Easy Import.

The foot is now posed so it fit into the shoes, but the rest pose has not changed. To fix this, press Apply Rest Pose. This is the same button that appears in the Corrections section, but is duplicated here for convenience.

We can not use the Retarget Armature Modifiers tool this time, because the Bardot skirt has extra bones that must be merged with the body rig. Instead we use the Merge Rigs tool in the Corrections section. Note that this time the armatures must be selected in the opposite order: clothes rig selected and body rig active. In the Merge Rigs option, deselect the Create Mesh Collection option.

 

Now we can pose the dressed character, and use the extra bones to keep the skirt from penetrating the legs.

As mentioned above, we cannot use the Copy Pose tool with the MHX or Rigify rigs, so we must convert the rig manually when everything else is done.

 

 

Voluminosity and Material Editor

The Daz Importer creates somewhat non-standard skin materials in order to make them as faithful to Daz Studio as possible. In particular, the BSDF method uses translucency and voluminosity nodes the match the translucency in Iray

This setup is quite non-standard for Blender skins, and may lead to longer render times, but in some cases it is necessary. The picture below shows Victoria 8, rendered in Blender with and without the voluminosity node.

However, it is quite easy to modify the skin to the right to something good-looking, using the material editor. Before importing the character, we need to turn off the Volume option in the global settings dialog, so the voluminosity nodes are not generated.

Modifying the node trees manually is quite cumbersome, because Daz skin consists of many materials, and we need to make changes consistently over all skin materials. Instead we can use the material editor, which has been improved recently. Start it by pressing Launch Material Editor in the Setup > Materials section. Before doing so, select the material that the editor uses as a blueprint. In this case we choose the Face-1 material.

At the top of the editor window we select the materials that will be affected by the changes. Clearly we only want to tweak the skin, lip and nail materials, and leave the eye and mouth materials unchanged. We can select the individual materials, but it is more convenient to use the Skin-Lips-Nails button to do this. The active material Face-1 is shown below the list of affected materials.

We then open the Translucency group, and change the Cycles Mix Factor to 1.0 and the SSS color to a light grey.

The mix factors determine whether the DAZ Translucent group uses translucency or sub-surface scattering, or a mixture of both. 0.0 means pure translucency, which works well together with the volume nodes, and 1.0 means pure SSS. Since we didn't generate any volume nodes, the Cycles Mix Factor is set to 1.0, i.e. pure SSS.

Finally we hit the Update Materials button. The result is not bad for a first attempt, considering that we only changed two parameters.

If we go back into the shader editor, we can see how the updates affect the node trees.

The Update Materials button does not close the material editor, so if we are not happy with the result we can continue to tweak the parameters. Here we changed the SSS color to a slightly reddish tone.