What are metaclasses in Python ?

Reprint from answer of Thomas Wouters in question What are metaclasses in Python? just for case that original answer unavaiable.


A metaclass is the class of a class. A class defines how an instance of the class (i.e. an object) behaves while a metaclass defines how a class behaves. A class is an instance of a metaclass.

While in Python you can use arbitrary callables for metaclasses (like Jerub shows), the better approach is to make it an actual class itself. type is the usual metaclass in Python. type is itself a class, and it is its own type. You won’t be able to recreate something like type purely in Python, but Python cheats a little. To create your own metaclass in Python you really just want to subclass type.

A metaclass is most commonly used as a class-factory. When you create an object by calling the class, Python creates a new class (when it executes the ‘class’ statement) by calling the metaclass. Combined with the normal __init__ and __new__ methods, metaclasses therefore allow you to do ‘extra things’ when creating a class, like registering the new class with some registry or replace the class with something else entirely.

When the class statement is executed, Python first executes the body of the class statement as a normal block of code. The resulting namespace (a dict) holds the attributes of the class-to-be. The metaclass is determined by looking at the baseclasses of the class-to-be (metaclasses are inherited), at the __metaclass__ attribute of the class-to-be (if any) or the __metaclass__ global variable. The metaclass is then called with the name, bases and attributes of the class to instantiate it.

However, metaclasses actually define the type of a class, not just a factory for it, so you can do much more with them. You can, for instance, define normal methods on the metaclass. These metaclass-methods are like classmethods in that they can be called on the class without an instance, but they are also not like classmethods in that they cannot be called on an instance of the class. type.__subclasses__() is an example of a method on the type metaclass. You can also define the normal ‘magic’ methods, like __add__, __iter__ and __getattr__, to implement or change how the class behaves.

Here’s an aggregated example of the bits and pieces:

    def make_hook(f):
    """Decorator to turn 'foo' method into '__foo__'"""
    f.is_hook = 1
    return f

class MyType(type):
    def __new__(mcls, name, bases, attrs):

        if name.startswith('None'):
            return None

        # Go over attributes and see if they should be renamed.
        newattrs = {}
        for attrname, attrvalue in attrs.iteritems():
            if getattr(attrvalue, 'is_hook', 0):
                newattrs['__%s__' % attrname] = attrvalue
            else:
                newattrs[attrname] = attrvalue

        return super(MyType, mcls).__new__(mcls, name, bases, newattrs)

    def __init__(self, name, bases, attrs):
        super(MyType, self).__init__(name, bases, attrs)

        # classregistry.register(self, self.interfaces)
        print "Would register class %s now." % self

    def __add__(self, other):
        class AutoClass(self, other):
            pass
        return AutoClass
        # Alternatively, to autogenerate the classname as well as the class:
        # return type(self.__name__ + other.__name__, (self, other), {})

    def unregister(self):
        # classregistry.unregister(self)
        print "Would unregister class %s now." % self

class MyObject:
    __metaclass__ = MyType


class NoneSample(MyObject):
    pass

# Will print "NoneType None"
print type(NoneSample), repr(NoneSample)

class Example(MyObject):
    def __init__(self, value):
        self.value = value
    @make_hook
    def add(self, other):
        return self.__class__(self.value + other.value)

# Will unregister the class
Example.unregister()

inst = Example(10)
# Will fail with an AttributeError
#inst.unregister()

print inst + inst
class Sibling(MyObject):
    pass

ExampleSibling = Example + Sibling
# ExampleSibling is now a subclass of both Example and Sibling (with no
# content of its own) although it will believe it's called 'AutoClass'
print ExampleSibling
print ExampleSibling.__mro__

Why always ‘ImportError: No module named ***’ ?

为什么在IDE中可以正常run起来的项目, 从命令行中启动就会报Import Error?

Python中,每个py文件被称之为模块(module),每个具有init.py文件的目录被称为包(package)。只要模块或者包所在的目录在Python的sys.path中,就可以使用import 模块或import 包来使用.
如果你要使用的模块(py文件)和当前模块在同一目录,只要import相应的文件名就好,比如在a.py中使用b.py:

import b

但是如果要import一个不同目录的文件(例如b.py)该怎么做呢? 首先需要使用sys.path.append方法将b.py所在目录加入到搜素目录中。然后进行import即可,例如

import sys
sys.path.append('c:\xxxx\b.py')`# 这个例子针对 windows 用户来说的 

大多数情况,上面的代码工作的很好。但是如果你没有发现上面代码有什么问题的话,可要注意了,上面的代码有时会找不到模块或者包(ImportError: No module named xxxxxx),这是因为sys模块是使用c语言编写的,因此字符串支持 ‘\n’, ‘\r’, ‘\t’等来表示特殊字符。所以上面代码最好写成:

sys.path.append('c:\\xxx\\b.py') 
或者sys.path.append('c:/xxxx/b.py') 

这样可以避免因为错误的组成转义字符,而造成无效的搜索目录(sys.path)设置。

sys.path是Python的搜索模块的路径集,是一个list, 可以在Python 环境下使用sys.path.append(path)添加相关的路径,但在退出Python环境后, 自己添加的路径就会自动消失了!

原因在于从IDE启动的项目或者模块, 会在运行时将当前工程的所有文件夹路径都作为包的搜索路径, 其本身已经将对应路径加入刀sys.path中了. Python在启动时, 会自动从sys.path中寻找路径来寻找对应的模块, 如果找不到,则会跑出ImportError.
所以, 从命令行中启动时, sys.path只包含Python自带的一些路径, 而没有加入自定义的模块/包的路径, 所以会导致

IDE :
['/home/bp/PycharmProjects/untitled1/dir2', '/home/bp/PycharmProjects/untitled1/dir2', '/home/bp/PycharmProjects/untitled1', '/home/bp/PycharmProjects/untitled1/dir1', '/home/bp/VirtualEnvs/tensorflow0625/lib/Python35.zip', '/home/bp/VirtualEnvs/tensorflow0625/lib/Python3.5', '/home/bp/VirtualEnvs/tensorflow0625/lib/Python3.5/plat-x86_64-linux-gnu', '/home/bp/VirtualEnvs/tensorflow0625/lib/Python3.5/lib-dynload', '/usr/lib/Python3.5', '/usr/lib/Python3.5/plat-x86_64-linux-gnu', '/home/bp/VirtualEnvs/tensorflow0625/lib/Python3.5/site-packages', '/usr/local/lib/Python3.5/dist-packages', '/usr/lib/Python3/dist-packages']

Terminal :
['', '/usr/lib/Python35.zip', '/usr/lib/Python3.5', '/usr/lib/Python3.5/plat-x86_64-linux-gnu', '/usr/lib/Python3.5/lib-dynload', '/usr/local/lib/Python3.5/dist-packages', '/usr/lib/Python3/dist-packages']

Solutions

常见的方式有三种
1. import
如果引用的文件和当前文件在一个文件夹下, 可以直接import

  1. sys.append

  2. *.pth
    在site-package下添加.pth文件, 在文件内用 绝对路径 标明引用路径.

将自己做的py文件放到 site_packages 目录下:

下面命令显示了 site-packages 目录:

python -c “from distutils.sysconfig import get_python_lib; print get_python_lib()”

但是这样做会导致一个问题,即各类模块都放到此文件夹的话,会导致乱的问题,这一点是显而易见的。

注意,也不创建子文件夹,再将自己的模块放到子文件夹解决问题,这会导致使用import 语句时错误。

使用pth文件,在 site-packages 文件中创建 .pth文件,将模块的路径写进去,一行一个路径,以下是一个示例,pth文件也可以使用注释:

# .pth file for the  my project(这行是注释)
E:\DjangoWord
E:\DjangoWord\mysite
E:\DjangoWord\mysite\polls

这个不失为一个好的方法,但存在管理上的问题,而且不能在不同的python版本中共享。

Boob!

上述的部分解释了引发ImportError的缘由, 这里给出一个可以通过递归添加项目目录下所有包路径的方法.

def append_sys_path(root_path=None):
    """
    递归添加项目下所有的Package路径.
    :return:
    """
    if root_path is None:
        pro_dir = os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
    else:
        pro_dir = root_path

    result_dir = [pro_dir]
    while result_dir:
        single_dir = result_dir.pop()
        init_flag = False
        subdir_list = []
        for single_path in os.listdir(single_dir):
            if single_path == '__init__.py':
                init_flag = True  # 根据当前目录是否存在__init__.py文件判断是否为package.
            single_path = os.path.join(single_dir, single_path)
            if os.path.isdir(single_path):
                subdir_list.append(single_path)
        if init_flag:
            sys.path.append(single_dir)
        result_dir.extend(subdir_list)

其他介绍ImportError的文章

  1. Python 101: All about imports
  2. Python导入模块的几种姿势 – 链接1的中文译文