shared.python.utils 1.0

Github link: shared.python.utils

Welcome to the second post on python utils. Lets get to it, shall we?

empty_list()

To make an empty list with a specific number of items looks like this:

# generates list full of None values: 
empty_list = [] * number_of_items

While its not terrible, its not great either. I made this function so I could fill the list with something specific. Like so:

empty_list = pyutils.empty_list(number_of_items=0, default_item=True)

time_it() and count_it()

These two are useful decorators. What is a decorator you ask? Think of it as writing a generic wrapper for a function, but you leave the function variable. If you want a step by step on how to make decorators, please visit my decorator discussion page.


get_sorted_by_most_common()

I suppose this is where we get specific to problems I’ve had. I needed to sort all items in a list by how many items there are. for example:

data = get_sorted_by_most_common(
    [
    "Mouth",
    "Corner",
    "Up",
    "Mouth",
    "Up",
    "Mouth",
    "Up",
    "Mouth",
    "Up",
    "Mouth",
    "Corner",
    "Up"
    ]
)

# result:
# ["Mouth", "Up", "Corner"]

I’m aware this is not a common thing, but hey, if you ever have a similar issue, here it is.


That’s mostly it. Please ask questions if you have any. Leave a comment. Let me know how I’m doing. Thank you all.

//Isoparm

shared.python.utils 1.1

Github link: shared.python.utils

Hello, and welcome to our first module. In this one we will make the most generic scripts we can think of. These are meant to help us in writing Python. They should not have any other dependencies other than Python or other well established Python libraries.


make_list()

This function converts any python object into a list(). This is particularly useful when dealing with packages such as Maya and 3DsMax. Many functions in those packages will return either a list() or a str(), depending on how many entries are returned. This is super annoying. You end up having to test the result to see if its a list() every time (ugh!). After writing that check seven times, I gave up.

Another added benefit of this function is that it can help you in making functions that take one or more items as arguments. For example:

# you can call a log function with a text:
log("Tell me something")

# or with an array:
log(["Tell me this", "Tell me that"])

Here is how that function would work:

import shared.python.utils as pyutils
 
# new_list == ["string"] 
new_list = pyutils.make_list("string")

# new_list == ["string"]
new_list = pyutils.make_list(["string"])

join_lists()

Have you noticed that some functions return False when they should return an empty string? Check this example:

results = list()

for item in list_of_items:
    results =+ imported.get_value(item)

Imagine that we dont own ‘imported’, its a imported library. And imagine that get_values() returns False if there are no values to get from item. If item ever has no values to return, you would get an error that looks like this:

TypeError: can only concatenate list (not "bool") to list

The solution is simple but annoying. After writing it several times, I decided to add it to my utils module.

This is how you would use it:

import shared.python.utils as pyutils

# new_list = ["one", "two", "three", "four"]
new_list = pyutils.join_lists(["one","two"],["three", False], ["four"])

remove_duplicates()

This one exists only for readability’s sake. Its not difficult to do this in a single line, but its hard-ish to read.

# its not easy to read this:
sans_duplicates = list(set(objects))

# this is much easier:
sans_duplicates = pyutils.remove_duplicates(objects)

are_items_in_list()

I find myself looking checking a list to see if it has a bunch of items. I do this often. You can use the set.intersection() function, but it’s hard to read. It ends up looking like this:

# how am I supposed to know what this means???
if len(set(full_list).intersection(items)):
    return

# much easier to read, I think.
if pyutils.are_items_in_list(items, full_list):
    reutrn

I suspect you begin to see a pattern. I always try to make code more readable. In many cases I over do it, I know. However, when I go back to my code weeks, months and years later, I don’t struggle with it so much. So I feel it’s worth it.

I think that’s enough for today. I will deal with more of this module on the next post.

Let me know in the comments if this is a helpful format. Also, if you have any questions about any of the modules mentioned, or any of the ones not mentioned, please dont hesitate.

As always, I welcome any suggestions.

// Isoparm

shared.python Intro

In my experience, having a good core library makes a world of difference. I couldn’t even imagine not having one. I owe all of my speed of development to my core library. It’s worth spending the time to get a good library started. And then, it’s worth keeping it up. I can’t stress this enough. To me, success in any Python project starts at your core library.

You will notice I use the words ‘core’ and ‘shared’ interchangeably. I dont like the word ‘core’ because it has baggage that I sometimes don’t fully follow and I end up spending a ton of time arguing semantics, rather than code. However, it’s an accepted word in the community, so it’s sometimes easier to just use it. I go back and forth on it. Maybe we should have a discussion on it in our discussions page?

Here are the modules that I’m currently planning for our core / shared library:

  • python
  • maya
  • math
  • ui
  • startup

python

We will start with the lowest level modules. These modules are meant to help with writing Python. For example, they can be wrappers for os or subprocess, or they can be a python utilities module for making some base decorators.

maya

These modules will contain wrappers for the maya.cmds, maya.mel, pymel, openMaya, and other Maya related modules. We will try to keep the code as un-opinionated as possible. Solutions here should be as generic as possible. So try to avoid thinking in terms of tools and workflow, and think more about APIs and what you think you will need for when you are making tools.

math

There are quite a few things that I don’t like doing in Maya. Math is one of them. If I can avoid opening Maya for my solutions, then I will be able to use my code in projects that are not for Maya. Among some of the modules in here, we will have vectors, angles, filters, math functions, point clouds, etc.

ui

This is one I end up arguing over a lot. In reality, this module is here for the sake of practicality, not for the sake of Pythonic purity. One thing I really like doing, is selecting a UI scheme that can work in most situations, and them making it so I can make GUIs quickly. I even developed an auto GUI maker once. I really liked it, since it allowed me to focus on the meat of the tools and code, and not on the GUI. As we develop this module further, I’m sure you will see the benefits.

startup

Yeah, we need one of these. It’s really convenient to have a single entry point for all your Python settings and environment. Here we will set up the logger, pick which Python Host we are in, load the correct versions of plugins, etc.

Alright, enough rambling. Next post, we will start on shared.python.utils.py

// Isoparm

Decorators & Contexts 101

Decorators and contexts are really useful. They let you wrap functions in pre-defined code. They can be incredibly useful for solving a wide array of issues. Here are some examples:

  • Timing how long a function takes to run.
    • I time functions all the time. Having a single line I can add to my function that will tell me how long it took is a great short cut.
  • Clean up Maya dirty scenes.
    • Sometimes you want to do something to your Maya scene that is destructive. Exporters do this all the time. The code could potentially leave the scene in a bad state, and you dont want to save that. You can add a context that saves the scene, does the job, and then restores the scene so it’s left as it was before you started.
  • Restore Maya settings.
    • Animators like to keep AutoKey on. Some tools don’t work well with that on. A decorator could make sure that the setting is off, run the function, and then turn it back on.
    • Batch process is slow if your viewport is updating for every file load. You can make a decorator that will stop updating the viewport while the batch process is happening.

See? They can be quite convenient. I wrote a file manager once that locked, copied, merged, and unlocked files on a server as users opened, changed, and closed those files. Without decorators, the code would have had a ton of repeated code that would then have to be maintained. I saved myself a monster headache by making decorators for all file operations.

Enough pontificating, let us make a context and a decorator.


time_it()

For this example we will write a context and a decorator for timing code and functions. This is a good example, since the solution is simple, and we can focus on the context and decorator.


1.1 Context

Step 1. write the code that solves the problem.

We want to time a block of code. This will solve that:

# record initial time:
time_start = timeit.default_timer()

# ---------------------------------
# code to be timed:
# ---------------------------------

# record time after code ran:
time_end = timeit.default_timer()

# inform the user:
print('Timed result : {0:.20f} sec'.format(time_end - time_start))

Step 2. Make it into a function:

We want to be able to import this code into other modules and run it. So it needs to be saved in a module as a function.

We need to import contextlib. It’s a pretty standard Python library, you should have no problems importing it.

import contextlib

@contextlib.contextmanager
def time_this():
	"""
	Context for timing blocks of code.
	"""
	
	# record initial time:
	time_start = timeit.default_timer()

	try:
		# Here is where/when the code to be timed will run:
		yield

	finally:
		# record time after code ran:
		time_end = timeit.default_timer()

		# inform the user:
		print('Timed result : {0:.20f} sec'.format(time_end - time_start))

To understand this function, you must understand two concepts: try/except/finally, and yield.

try/finally:

It’s not too complicated. You are asking Python to “try” to run some code. If it fails, it will skip the failed code and continue running the rest of the function. Then, “finally” will always run, regardless of the success of “try”. So in our example, regardless of weather or not the code evaluates correctly, the function will tell you how long it took.

yield:

This one is a bit tricky. I think I’ll do a much more in-depth discussion about yield at a later date. For now, assume that “yield” passes the evaluation stream to the code you want to time. It’s like driving. You yield to a pedestrian, which makes you stop, so he can start. When he is done crossing the street, you start up where you left off (you dont go back home and start the trip again). It’s not exactly this, but for now it will do.

So how do we use this function? It’s not hard. Once you have it written, you can use it all over the place. You can even forget how to time scripts, now.

from time_module import time_this

with time_this:

	# ---------------------------------
	# code to be timed:
	# ---------------------------------

1.2 Decorator

Contexts are great, but you have to write them in every time you want to use them. What if we could tell Python, that every time it calls a specific function, it would time it? This way we would not have to write the with context statement every time.

This is what decorators are for. Granted, timing a function is not a great use of a decorator, but it’s not a bad one either. Lets use it.

Step 1. Write the code that solves the problem.

We did that already on the example above. So, moving along…

Step 2: Make a function out of it.

For the same reasons as above (importing from another module), we need a function. This function, however, will be a bit different.

Instead of having code to run, a decorator evaluates a function every time that function is called. Think of it as a wrapper for the function, except the function is variable. We need to be able to tell the wrapper what function to evaluate. Confusing?

Ok, let me try again.

We are going to tell Python that every time we call function A(), we want it to print how long it took to evaluate. We are also telling Python that we want the same treatment for functions B(), C() and D(), but we only want to write the wrapper once.

So first things first, we need a function that accepts a function as an argument:

def time_it(func):

If you have written wrappers before, you know that you need to capture the return from the main function and return it in the wrapper. If you dont do this, you lose the data you got from the function.

To do this, we define a function inside the wrapper that will capture that result.

def time_it(func):

    # with this function, we capture and return the return value:
    def timed():

        # function to be timed:
        result = func()

        return result

    return timed

As you can see, we are storing the return of the function in a variable called result. Then we return result.

We need time_this() to evaluate func, right? But what if func has arguments? I mean func can be any function at all, with any number and style of arguments. So how do we deal with that?

Simple, *args and *kwargs are your friends here.

def time_it(func):

    # with this function, we capture and return the return value:
    # add args and kwargs to be able to pass any and all args of the func:
    def timed(*args, **kwargs):

        # function to be timed:
        result = func(*args, **kwargs)  # pass the arguments to the function

        return result

    return timed

Great, we now have a decorator that calls the function it passes, and it handles the arguments well. Unfortunately, it does nothing else. So lets add the timing code to it:

def time_it(func):

    # with this function, we capture and return the return value:
    # add args and kwargs to be able to pass any and all args of the func:
    def timed(*args, **kwargs):
        
        # record initial time:
        time_start = timeit.default_timer()

        # function to be timed:
        result = func(*args, **kwargs)  # pass the arguments to the function

        # record time after code ran:
        time_end = timeit.default_timer()

        # inform the user:
        print('Timed func : {}'.format(func.__name__))
        print('Timed result : {0:.20f} sec'.format(time_end - time_start))
        
        return result

    return timed

This function should work now for you now. But how do we use it? I really like how this is used. Once ready, check out how easy it is to use a decorator:

from time_module import time_it

@time_it  # <- Right here. Add this to any function you want timed.
def my_function(args):
	print args

Adding the line @time_it above any function declaration will tell Python you want it timed every time it runs. If you were to call my_function(“hello”) in a for loop, you would get a “hello” for every iteration of the loop.


And that’s mostly it. while there are more things we can discuss, this is a good intro to decorators and contexts. I hope this helps. Please leave me a comment with questions, corrections, suggestions, or ideas on how to make this page better. Thank you!

// Isoparm

Discussion – Python Readability

Readability > purity, and sometimes, even practicality. I have a pretty bad memory. When it comes to code it lasts about 2 months (when I’m lucky). This means that if I write a module in June, by September I can’t tell you how the module works, just what it does.

I love writing code. I hate reading code. I had a mentor tell me once that it takes 100% of your brain to write a module, but that reading that module is much harder. He was right. He was so right, in fact, that reading code for me is almost an exercise in futility. I curse at the person who wrote it. I hate reading code.

I have to take steps to make sure I CAN read code. For years I didn’t care about readability and I paid the price in frustrations. One fine day, after I had written a particularly difficult module (for me, at the time), I tried reading it as if I had encountered it for the first time. I started making changes, simple changes, to make it easier to read.

About 4 months later, I had to fix a bug on it. I opened it and I was able to read it!

Now, I go out of my way to make my code as readable as I can. I assume that when I get back to it, it will be in 10 years, and I’ll be half drunk. That way, when I do get back to it, I can read it and understand it quickly.

I also believe that the code itself needs to be readable, not just well commented. Comments are great and I will swear by them all day long, however, I need the code to be readable or comments will be completely lost on me.

Some examples:


# this is not very readable (what is an "isdir" anyway): 
if os.path.isdir("C:\file_path"): 

# this feels better, i think: 
if pyfile.is_dir("C:\file_path"): 

Not convinced? Ok, next example: getting the influence list from a skinned mesh in Maya.

Putting aside the fact that you have to feed it a skinCluster instead of a mesh, the default Maya command is still confusing. Since the commands can be used in query mode or command mode, I end up having to scan the code for “query=”. I dont like it.

# Maya default: 
influences = cmds.skinCluster(cluster, query=True, influence=True) 

# how do you feel about this one? 
influences = myskin.influences(mesh) 

Still not convinced? Lets try to make a Maya reference.

# honestly, I can't read this. 
cmds.file("C:\file_path", reference=True, ignoreVersion=True, namespace=prefix) 

# That's much better. 
reference.create("C:\file_path", namespace) 

If this last example does not convince you, nothing will.

# get the name of a file: 
name = os.path.basename(os.path.splitext(file_path)[0])  #WTF?! 

# or: 
name = pyfile.name(file_path)

Trust me on this, your future self will be very thankful that you spent the time making sure he could read your chicken scratches.

So, what do you think? Comment away, and we can have a fun discussion.

// Isoparm

Env Setup – Part 1.0 – Software Needed For Maya/Python Development

First off, we need to have an appropriate environment. In order for us to be set up for success, we need to well… set ourselves up for success. I spent years using the “wrong” IDE. I fought with it, I yelled at it, i even re installed it a few times thinking it would fix my problems. For the longest time i was convinced that the problem was me, not the software. While there are many right answers to the question of the “Best IDE for Python” questions, there are definitely some wrong ones. So don’t be like me, pick a better one.

Here is the list of software we need. Some of it costs money… well it all costs money, but some of it can be used in trial, education, or limited versions. I will attempt to have an accessible environment during this project, so we can all speak the same language.

  1. Maya. Well sure, this is a given, but I had to put it in here. The software is not exactly cheap, but there are some things you can do. If you are a student enrolled in a school, you probably have access to an education version (that one will do nicely). You can also try out the software for 30 days (we probably wont be finished by then, but 30 days should be enough to figure out if you want to continue).
  2. Python 2.7. Unfortunately, Autodesk has not moved on from Python 2.x to 3.x. This means that we need to do our work in an older version of Python. Since Autodesk is pretty much a monopoly when it comes to rigging, then we must make do. Make sure you download 2.7.x, and not 3.x.
  3. PyCharm. This is my go-to IDE for Python. It integrates nicely with Maya and with Github.
  4. Sublime Text. Why have 2 IDEs? Well, there are some nice things about having a light weight text editor that is fully featured. I use it quite a bit. If you are on a  tight budget, you can skip this one.
  5. Github account. I will share all the code written for this project in Github, so it will be helpful to have an account so you can follow along. To setup Git, follow these instructions: Set Up Git.

OK, now go install all the software and come back, I’ll wait.

// Isoparm

Next: Connect Pycharm and Github