Scripting Improvements

I have spent a log of time developing the DAZ Importer plugin, but I rarely use its scripting capabilities. However, every button becomes an operator that can be called from a script; a list of operators defined by the DAZ Importer can be found here. These operators are defined automatically and usually work out of the box, but there are some caveats regarding operators that act on multiple files, morphs or shapekeys.

Assume that we want to import a figure with Easy Import. We select the duf file (or the corresponding .png or .tip.png file) and press Easy Import DAZ. To do the same thing programatically, we could expect that the following script would work: 

bpy.ops.daz.easy_import_daz(
    filepath = "D:/home/scenes/victoria-rocker.duf"
)

However, nothing is imported. The reason is that the Easy Import tool can import several files at once. If we select the following three files, three different figures will be imported at once.

And here are the figures in Blender.

Importing multiple character in one stroke may not seem very useful, but we may want to import thirty different props from a single directory, and then it is easier to ímport thirty files at once than to press the Easy Import button thirty times and only import a single prop each time.

The old way to import multiple files uses the api.set_selection function, which needs to be called before easy_import_daz is invoked. Functions defined by the DAZ Importer plugin are documented here.

import bpy
from bl_ext.user_default.import_daz import api

api.set_selection([
    "D:/home/scenes/aiko-basic-wear.duf",
    "D:/home/scenes/victoria-rocker.duf",
    "D:/home/scenes/michael-basic-wear.duf"])
bpy.ops.daz.easy_import_daz()

Unfortunately, this does not work very well. Under the hood the set_selection function sets a global variable, and this leads to complications for operators that invoke other operators, like easy_import_daz does. Recently I learned from Rakete that there is another way to access multiple files which does not involve any global variables.

When a file selector exists, it sets two keyword arguments: the StringProperty "directory" and the CollectionProperty "files". An alternative way to import multiple files is like this:

bpy.ops.daz.easy_import_daz(
    directory = "D:/home/scenes",
    files = [{"name" : "aiko-basic-wear.duf"},
             {"name" : "victoria-rocker.duf"},
             {"name" : "michael-basic-wear.duf"}])

Actually, importing files in this way has always worked, but I was not aware of it. The "files" argument is a bit complicated; it is a list of dicts with the "name" keyword, which is how we specify a CollectionProperty in scripts.

To import a single file we simply let "files" be a list with a single argument.

bpy.ops.daz.easy_import_daz(
    directory = "D:/home/scenes",
    files = [{"name" : "victoria-rocker.duf"}])
    
This way of importing multiple files is not limited to Easy Import. The following script imports three poses to the active object. The second line turns on automatic keying.

import bpy
bpy.context.scene.tool_settings.use_keyframe_insert_auto = True
folder = "/people/genesis 8 female/poses/base poses/"
files = ["base pose kneeling a.duf",
         "base pose kneeling b.duf",
         "base pose kneeling c.duf"]         
bpy.ops.daz.import_pose(
   directory = api.get_absolute_path(folder),
   files = [{"name" : file} for file in files],
)

This script imports three poses to consequtive frames. 

Many tools opens another type of selector, which lets us select which morphs to import, which shapekeys to transfer, which materials to modify, etc. Such selections used to be made with the api.set_selection function as well, but now the preferred way is to use the "selection" keyword. Again the argument is a CollectionProperty, which corresponds to a list of dicts with the "name" keyword in scripts.

The following script import three units morphs:

files = ["eCTRLEyesClosed.dsf",
         "eCTRLEyesClosedL.dsf",
         "eCTRLEyesClosedR.dsf"]
bpy.ops.daz.import_units(
    selection = [{"name" : file} for file in files]
)

It is thus equivalent to the following selection in the Import Units dialog.

If the "selection" keywprd is omitted, or equivalently if selection = [], all elements are selected. Thus the line

bpy.ops.daz.import_visemes()

imports all available viseme morphs. We can then set the corresponding rig properties.

rig = bpy.context.object
rig["eCTRLvAA"] = 1.0
rig["eCTRLvOW"] = 0.4

Mathjax Test

The formulas in the header.

$$\begin{array}{rl}[{\mathcal{L}}_{\xi },{\mathcal{L}}_{\eta }]& ={\mathcal{L}}_{[\xi ,\eta ]}+\frac{1}{2\pi i}\int dt{\dot{q}}^{\rho }(t)(c_1{\partial }_{\rho }{\partial }_{\nu }{\xi }^{\mu }(q(t)){\partial }_{\mu }{\eta }^{\nu }(q(t))+\\ & +c_2{\partial }_{\rho }{\partial }_{\mu }{\xi }^{\mu }(q(t)){\partial }_{\nu }{\eta }^{\nu }(q(t))),\\ [{\mathcal{L}}_{\xi },q^{\mu }(t)]& ={\xi }^{\mu }(q(t)),\\ [q^{\mu }(t),q^{\nu }(t^{\prime })]& =0.\end{array}$$

If you use Firefox with NoScript, as I do, you need to whitelist jsdelivr.net and polyfill.io, otherwise only the uncompiled LaTex code will show up.

If anybody wonders, this is the Virasoro-like extension of the diffeomorphism algebra in multiple dimensions. e.g. on the $d$-dimensional torus. When $d=1$, both terms proportional to $c_1$ and $c_2$ reduce to the ordinary Virasoro algebra on the circle, a piece of mathematics that is popular e.g. in string theory and statistical physics. By the way, I discovered the $c_2$ term myself a long time ago, which is something that I am still proud of ($c_1$ was discovered by Rao and Moody).

When I started this blog almost a decade ago I had planned to write about this and how it in my opinion fits into physics, hence the name of the blog. I never got around to do that, but that might change in the future.

 

Adding Extra Bones to Rigify

Some character have non-standard bones, e.g. a rigged hair. When we convert such a rig to Rigify, the rigified rig contains those bones, but they may be inconvenient to pose individually.

 

Here we have a figure with a rigged ponytail. The ponytail bones belong to the Custom collection.,

And here the rig has been converted to Rigify, with the hair bones still in the Custom collection. Posing each of hair bone individually can be done but is inconvenient.There are a few things that we can do to make it easier to pose the ponytail.

We can Add IK Goals. This tool is located in the Rigging > Chains panel which requires that the Rigging Tools feature is enabled.

Alternatively, we can Add Winders.

However, Rigify is a modular rigging system and we may want to use some of the Rigify types. Previously that has not been so easy to do. In principle we can first generate the metarig, then add the custom bones to it, and finally rigify the modified metarig. However, that requires further postprocessing because the vertex groups must be renamed. In the last version of the DAZ Importer there is a much simpler way. Just select a bone in the original DAZ rig and add a Rigify type to it.

Let us give the ponytail the control of a basic Rigify tail. Select the bone that we want to be the root of the tail, and assign a suitable Rigify type to it. Here we choose spines.basic_tail.

The Rigify Type panel contains various option for the bone chain. By default a basic tail can only bend along the X axis.

Once a bone has been assigned a Rigify type, it is added to the metarig together with its descendants. Any parent bones that are not part of the standard Rigify armature are also added.

Now the rigified armature has a ponytail with the extra control bones.

The entire ponytail can now be posed by rotating the target bone at the end of chain.

MHX Properties are now ID Properties

As discussed in the post on Custom Properties Cleanup, custom properties used by the DAZ Importer are now combined into a single, complex property. However, there are two exceptions:

1. Morph properties, because they are part of a web of drivers that would not be evaluated in the correct order if they were parts of a complex property.
2. Properties that need to work when the add-on is disabled, e.g. at a render farm.
   

 The custom properties used by the MHX rig belong to the second category. 

There are two types of custom properties in Blender:RNA properties and ID properties, or properties that are defined by the system or by add-ons and those that are created on the fly. The problem with RNA properties is that they do not work properly if the add-on is disabled. MHX properties have nevertheless been implemented as RNA properties, because ID properties did not worked with file linking in some earlier  version of Blender. This problem has been fixed in recent Blender versions, and therefore there is no reason to use RNA properties anymore.

This change should be backwards compatible, almost. Even if the MHX rig was created in an earlier version of the DAZ Importer, where the MHX properties were RNA properties, it should still function correctly with the latest version. This works because RNA properties are automatically converted to ID properties if the API no longer defines them. The only thing that does not work is animations with dynamic FK/IK switching.

Here are the custom properties of a recent MHX rig. The MHX properies begin with "Mha" (there was a reason why they don't start with "Mhx", but I don't remember why anymore). Note the cogwheel to the right of the property value. This indicates that we deal with ID properties rather than RNA properties, which would have "API Defined" written to the right instead. 

Pressing the cogwheel brings up the Edit Property window. Note that MHX properties are Library Overridable at the bottom of the window.

Save the scene with the MHX rig, open a new file, and link the collection with the MHX character into the new scene. To make the character useful in the new scene, choose Object > Library Override > Make.

We can still access the custom properties, but they can not be edited. When we change a linked property it turns green.

Now do the following steps:

1. Create an animation and save the file.
2. Disable the DAZ Importer and MHX RTS add-ons.
3. Restart Blender.
   

 We have now recreated the situation at a render farm, where the DAZ Importer and MHX RTS are not available. Nevertheless the animation works correctly, including the animation of custom properties needed e.g. for FK/IK switching.

Custom Properties Cleanup

The DAZ Importer creates lots of custom properties when a character is imported. These properties contain information from the duf file which is necessary for the proper function of other tools.

The add-on does not only create lots of object properties, but custom properties for meshes, materials, bones and posebones as well.

However, spamming the area with lots of custom properties is considered impolite and is frown upon. Other add-ons such as Rigify or MMD tools create a single custom property which contains all the needed information in a complex data structure.

Rigify only creates one object property and one armature property.

 

In the last commit the custom properties have been redesigned, and now the DAZ Importer also adds a single custom property. This probably comes with a small performance penalty, because accessing the data of a complex property takes more time than accessing simple properties. However, this effect should be neglible.

Care has been taken to make this change backward compatible, so old files should still work without modification. However, it is not forward compatible, so figures imported with the latest version will not work propertly in older versions of the DAZ Importer.

The Utilities panel has a new subpanel, called DAZ Importer Properties, where the components of the custom properties can be inspected. This is mainly for debugging purposes. The listing also indicates whether objects use the modern, single, complex property, and the legacy multiple, simple properties.

However, there are some situations where simple properties are still needed. One is if the figure should work correctly even if the DAZ Importer add-on is disabled or not even installed. A typical situation is if we make an animation with Simple IK and FK-IK switching. The animation should still work properly if the file is sent to a render farm which has not installed the DAZ Importer. This will work because the Simple IK properties are simple properties.

The MHX rig does not work by default if the MHX runtime system is disabled. The reason is that MHX uses RNA properties rather than ID properties, for reasons explained here. To make it work at a render farm the file properties.py  in the runtime folder must be saved with the blend file, in the same way as the file morph_armature.py must be loaded to make morping armatures work without the DAZ Importer. This is inconvenient and I intend to replace MHX RNA properties by ID properties in the future.

Finally, there is an important type of custom properties that must be simple propertes: morphs. In Blender morphs are implemented as object and armature properties that drive bone rotations, shapekeys and other properties. Morph properties can not be part of a single, complex property. That would slow down the evaluation of morphs, which already can take excessive time if many morphs are loaded. Even more importantly, it wouldn't work at all, because the morphs might evaluated in the wrong order. 

Morph properties are connected with a complex web of drivers, and it is crucial that the morphs are evaluated in the right order. This is guaranteed if the morph properties are simple properties.. There would be no such guarantee if they were parts of a complex property, and in fact that is not always the case. Besides, the morphs should work if the DAZ Importer is not enabled, otherwise we would end up with ugly faces.

So unfortunately the add-on is still flooding the custom properties if a figure has many morphs. I don't see any way around that.

Viewport Performance

DAZ meshes can be quite heavy and can slow down the viewport. Genesis 9 is worse than older generations, but clothes and especially hair can have a much higher poly-count. Add subdivisions, hair, drivers, etc. and viewport performance could become much worse.

Here are some suggestions for improving viewport performance. The suggestions are discussed in more detail below.

• Use solid view instead of rendered view.
  
• Turn on simplify and set max viewport subdivisions to zero.
• Disable drivers when when you pose, enable them when you work with morphs. The plugin can easily create hundreds of drivers that take a lot of time to update (Morphs > Disable Drivers).
• Add shapekey mute drivers to disable shapekeys with zero value.
  
• Put heavy meshes in separate collections that can be excluded during posing. Hair can be especially heavy.
• Put objects that are not in view in separate collections.
• Finalize meshes and armature in the Finishing panel. This removes some data that is needed for adding morphs etc., but improves viewport performance.
• Add a mannequin to block the animation. This a collection of meshes that are bone parented, which performs much better than a mesh with vertex groups.

This post will be updated include other optimization techniques.

Solid view instead of rendered view

This is sort of obvious. Even textured mode is more costly than material mode.

Simplify

The Simplify tab contains Blender's native tools for improving viewport speed. Usually we don't need to see subdivisions in the viewport. There is a global setting that limits the number of subdivision levels when DAZ scenes are imported, both the viewport and render levels.

 

The texture limit speeds up rendered view. However, the large textures may still take a lot of time to load, because Blender has to load the full texture before it can apply limitations. The Resize Textures tool permanently replaces textures with lower-resolution ones. Note that textures must be saved locally before they can be resized.

 

Disable drivers

Morphs are implemented in Blender as a complex web of drivers that control bone rotations and locations, mesh shapekeys, and custom properties. A DAZ figure with the standard morphs loaded can easily have many hundreds of drivers, and the evaluation of many drivers takes significant time.

The Disable Drivers button at the top of the Morphs panel disables all morph drivers, thus eliminating the need to compute the drivers. Shapekeys are also muted.

When the drivers are disabled the morphs do not function, and the Morphs panel only contains a button to Enable Drivers again

 

Shapekey mute drivers

DAZ figures often have many shapekeys, e.g. for JCMs and FACS morphs. The evaluation of those shapekeys takes time, but it is unnecessary to include zero shapekeys in the process.

The global option "Shapekey Mute Drivers" adds drivers to the shapekey enable channel, muting the shapekey if it is zero (within rounding errors). Muted shapekeys are not included in the evaluation of the final vertex locations which should lead to a performance gain.

When "Optimize JCM Drivers" is enabled some drivers related to JCMs are eliminated.

 

Separate collections for different meshes

Hair and clothes are often heavier than the figure itself, but are not really necessary when posing. We can put such objects into subcollections that are disabled during posing and only enabled again when it is time for rendering.

Here is Aiko with hair and clothes. Hair and clothes have been put into separate collections.

Here the hair and clothes collections are disabled. This does not really hamper posing.

 

Finalize meshes and optimize drivers 

The Finishing panel contains some buttons that can improve performance. However, this should be done as a final step when the figure setup is finished.

• Finalize Meshes: Removes internal properties from meshes, needed e.g. to merge geografts.
• Optimize Drivers: Optimize the web of drivers. New morphs can not be loaded afterwards. 
  

The Final Optimizations option in Easy Import tool does the same thing.

 

Mannequins

A mannequin is a collection of meshes that are parented to the bones of the armature. This improves performance because bone parenting can be evaluated much faster than the influence of multiple vertex groups. Mannequins are useful for blocking out an animation, even if final tweaking with the real mesh may be necessary. The DAZ Importer can create a mannequin for a selected set of meshes.

To create a mannequin, select the meshes and press "Add Mannequin". This tool requires that the Object Tools feature is enabled.

The original meshes are put into a subcollection called "Meshes", and the mannequin meshes belong to the new "Mannequin" collection. We can now pose the figure with the mannequin enabled, and switch to the original meshes for final tweaking and rendering

 

FILATOON

With daz studio 4.23 comes Filatoon, a new shader to draw toon style figures. With blender 4.3 and the new 4.3 version of the importer we can now get filatoon figures in blender. Beweare this is a first release so not all the filatoon features are supported, but it is enough to import most toon figures available in the DAZ shop.

ref. https://bitbucket.org/Diffeomorphic/import_daz/issues/2235/

Note that filatoon can only be rendered with eevee, not with cycles, this is the same in daz studio where filatoon only works with filament and not with iray. The toon outline is rendered with freestyle.

However there's a difference in lighting between daz studio and blender, as toon figures in daz studio are not lit by the HDRI, while in blender they always are. Even with the new light linking in blender 4.3 there's no way to exclude the HDRI. Of course this is not a issue if we only use regular lights without HDRI.

Below there's an example where we import a scene with HDRI, we have the G8F figure with iray materials, and the G9F toon with filatoon materials. We can see that in daz studio iray can't correctly render the toon figure, while filament can't correctly render the G8 figure. In blender we can render with eevee and both the toon and G8 figures are rendered correctly, however both are affected by the HDRI.

We can adjust both the HDRI threshold and the toon light inside the "daz toon diffuse" and "daz toon light" groups, note that adjusting one group will affect the whole scene.

update. OUTLINE MODE. Due to some user requests Thomas implemented different outline methods, or not importing the outline at all. The option is in the global settings.

The viewport in solid mode needs the backface culling for solidify to work, this is done automatically by the addon if "render settings" is set to "update" in the global settings. Note that solid mode can't correctly render filatoon anyway.

The viewport in material preview or rendering using eevee works fine and that's what the filatoon shaders are for.

• None: don't import the daz outline.
• Freestyle: use freestyle, visible only in rendering not in the viewport.
• Line Art: use the grease pencil, visible in the camera view and rendering.
• Solidify: use the solidify modifier, visible in the viewport and rendering.