标签归档:mysql

设置Django以使用MySQL

问题:设置Django以使用MySQL

我想稍微远离PHP,学习Python。为了使用Python进行Web开发,我需要一个框架来帮助模板和其他事情。

我有一台非生产服务器,用于测试所有Web开发内容。这是一个运行MariaDB而不是常见的MySQL服务器软件包的Debian 7.1 LAMP堆栈。

昨天我安装了Django并创建了我的第一个项目firstweb。我尚未更改任何设置。

这是我的第一个大困惑。在教程中,我跟随那个家伙安装了Django,开始了他的第一个项目,重新启动了Apache,从那时起Django就开始工作了。他转到浏览器,然后毫无问题地转到Django默认页面。

但是我,我必须进入我的firstweb文件夹并运行

python manage.py runserver myip:port

而且有效。没问题。但是我想知道它是否应该像这样工作,并且这是否会引起问题?

我的第二个问题是我想对其进行设置,以便它使用我的MySQL数据库。我进入/ firstweb / firstweb下的settings.py,看到了ENGINE和NAME,但不确定在这里放什么。

然后在USER,PASSWORD和HOST区域中,这是我的数据库及其凭据吗?如果我使用本地主机,是否可以将本地主机放在HOST区域中?

I want to move away from PHP a little and learn Python. In order to do web development with Python I’ll need a framework to help with templating and other things.

I have a non-production server that I use to test all of web development stuff on. It is a Debian 7.1 LAMP stack that runs MariaDB instead of the common MySQL-server package.

Yesterday I installed Django and created my first project called firstweb. I have not changed any settings yet.

Here is my first big piece of confusion. In the tutorial I followed the guy installed Django, started his first project, restarted Apache, and Django just worked from then on. He went to his browser and went to the Django default page with no problems.

Me however, I have to cd into my firstweb folder and run

python manage.py runserver myip:port

And it works. No problem. But I’m wondering if it is supposed to work like this, and if this will cause problems down the line?

My second question is that I want to set it up so it uses my MySQL database. I go into my settings.py under /firstweb/firstweb and I see ENGINE and NAME but I’m not sure what to put here.

And then in the USER, PASSWORD, and HOST areas is this my database and its credentials? If I am using localhost can I just put localhost in the HOST area?


回答 0

MySQL支持很容易添加。在您的DATABASES字典中,您将有一个像这样的条目:

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.mysql', 
        'NAME': 'DB_NAME',
        'USER': 'DB_USER',
        'PASSWORD': 'DB_PASSWORD',
        'HOST': 'localhost',   # Or an IP Address that your DB is hosted on
        'PORT': '3306',
    }
}

从Django 1.7开始,您还可以选择使用MySQL 选项文件。您可以这样设置DATABASES数组来完成此操作:

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.mysql',
        'OPTIONS': {
            'read_default_file': '/path/to/my.cnf',
        },
    }
}

您还需要/path/to/my.cnf使用与上面类似的设置来创建文件

[client]
database = DB_NAME
host = localhost
user = DB_USER
password = DB_PASSWORD
default-character-set = utf8

使用Django 1.7中的这种新连接方法,重要的是要知道建立了顺序连接:

1. OPTIONS.
2. NAME, USER, PASSWORD, HOST, PORT
3. MySQL option files.

换句话说,如果在OPTIONS中设置数据库的名称,它将优先于NAME,而NAME将覆盖MySQL选项文件中的所有内容。


如果您只是在本地计算机上测试应用程序,则可以使用

python manage.py runserver

添加ip:port参数允许您自己的机器以外的其他机器访问您的开发应用程序。准备好部署应用程序后,建议您阅读djangobook上有关部署Django章节

MySQL默认字符集通常不是utf-8,因此请确保使用以下sql创建数据库:

CREATE DATABASE mydatabase CHARACTER SET utf8 COLLATE utf8_bin

如果您正在使用Oracle的MySQL的连接器ENGINE线应该是这样的:

'ENGINE': 'mysql.connector.django',

请注意,您首先需要在操作系统上安装mysql。

brew install mysql (MacOS)

此外,mysql客户端软件包已针对python 3进行了更改(MySQL-Client仅适用于python 2)

pip3 install mysqlclient

MySQL support is simple to add. In your DATABASES dictionary, you will have an entry like this:

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.mysql', 
        'NAME': 'DB_NAME',
        'USER': 'DB_USER',
        'PASSWORD': 'DB_PASSWORD',
        'HOST': 'localhost',   # Or an IP Address that your DB is hosted on
        'PORT': '3306',
    }
}

You also have the option of utilizing MySQL option files, as of Django 1.7. You can accomplish this by setting your DATABASES array like so:

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.mysql',
        'OPTIONS': {
            'read_default_file': '/path/to/my.cnf',
        },
    }
}

You also need to create the /path/to/my.cnf file with similar settings from above

[client]
database = DB_NAME
host = localhost
user = DB_USER
password = DB_PASSWORD
default-character-set = utf8

With this new method of connecting in Django 1.7, it is important to know the order connections are established:

1. OPTIONS.
2. NAME, USER, PASSWORD, HOST, PORT
3. MySQL option files.

In other words, if you set the name of the database in OPTIONS, this will take precedence over NAME, which would override anything in a MySQL option file.


If you are just testing your application on your local machine, you can use

python manage.py runserver

Adding the ip:port argument allows machines other than your own to access your development application. Once you are ready to deploy your application, I recommend taking a look at the chapter on Deploying Django on the djangobook

Mysql default character set is often not utf-8, therefore make sure to create your database using this sql:

CREATE DATABASE mydatabase CHARACTER SET utf8 COLLATE utf8_bin

If you are using Oracle’s MySQL connector your ENGINE line should look like this:

'ENGINE': 'mysql.connector.django',

Note that you will first need to install mysql on your OS.

brew install mysql (MacOS)

Also, the mysql client package has changed for python 3 (MySQL-Client works only for python 2)

pip3 install mysqlclient

回答 1

首先,请运行以下命令以安装python依赖项,否则python runserver命令将引发错误。

sudo apt-get install libmysqlclient-dev
sudo pip install MySQL-python

然后配置#Andy定义的settings.py文件,并在最后一次执行:

python manage.py runserver

玩得开心..!!

To the very first please run the below commands to install python dependencies otherwise python runserver command will throw error.

sudo apt-get install libmysqlclient-dev
sudo pip install MySQL-python

Then configure the settings.py file as defined by #Andy and at the last execute :

python manage.py runserver

Have fun..!!


回答 2

如果您使用的是python3.x,则运行以下命令

pip install mysqlclient

然后像这样更改setting.py

DATABASES = {
'default': {
    'ENGINE': 'django.db.backends.mysql',
    'NAME': 'DB',
     'USER': 'username',
    'PASSWORD': 'passwd',
  }
  }

If you are using python3.x then Run below command

pip install mysqlclient

Then change setting.py like

DATABASES = {
'default': {
    'ENGINE': 'django.db.backends.mysql',
    'NAME': 'DB',
     'USER': 'username',
    'PASSWORD': 'passwd',
  }
  }

回答 3

如上所述,您可以轻松地首先从https://www.apachefriends.org/download.html安装xampp, 然后按照以下说明进行操作:

  1. http://www.unixmen.com/install-xampp-stack-ubuntu-14-04/安装并运行xampp ,然后从GUI启动Apache Web Server和MySQL数据库。
  2. 您可以根据需要配置Web服务器,但默认情况下Web服务器位于http://localhost:80,数据库位于port 3306,而PhpMyadmin位于http://localhost/phpmyadmin/
  3. 从这里您可以看到您的数据库,并使用非常友好的GUI访问它们。
  4. 创建要在Django项目上使用的任何数据库。
  5. settings.py像这样编辑文件:

    DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.mysql',
        'NAME': 'DB_NAME',
        'HOST': '127.0.0.1',
        'PORT': '3306',
        'USER': 'root',
        'PASSWORD': '',
    }}
  6. 在virtualenv中安装以下软件包(如果您在virtualenv上使用django,则更可取):

    sudo apt-get安装libmysqlclient-dev

    pip安装MySQL-python

  7. 而已!!您已经以非常简单的方式为MySQL配置了Django。

  8. 现在运行您的Django项目:

    python manage.py迁移

    python manage.py运行服务器

As all said above, you can easily install xampp first from https://www.apachefriends.org/download.html Then follow the instructions as:

  1. Install and run xampp from http://www.unixmen.com/install-xampp-stack-ubuntu-14-04/, then start Apache Web Server and MySQL Database from the GUI.
  2. You can configure your web server as you want but by default web server is at http://localhost:80 and database at port 3306, and PhpMyadmin at http://localhost/phpmyadmin/
  3. From here you can see your databases and access them using very friendly GUI.
  4. Create any database which you want to use on your Django Project.
  5. Edit your settings.py file like:

    DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.mysql',
        'NAME': 'DB_NAME',
        'HOST': '127.0.0.1',
        'PORT': '3306',
        'USER': 'root',
        'PASSWORD': '',
    }}
    
  6. Install the following packages in the virtualenv (if you’re using django on virtualenv, which is more preferred):

    sudo apt-get install libmysqlclient-dev

    pip install MySQL-python

  7. That’s it!! you have configured Django with MySQL in a very easy way.

  8. Now run your Django project:

    python manage.py migrate

    python manage.py runserver


回答 4

实际上,不同的环境,python版本等等存在很多问题。您可能还需要安装python dev文件,因此要“强行安装”,我将运行所有这些文件:

sudo apt-get install python-dev python3-dev
sudo apt-get install libmysqlclient-dev
pip install MySQL-python
pip install pymysql
pip install mysqlclient

您应该接受公认的答案。如果对您很重要,可以删除不需要的软件包。

Actually, there are many issues with different environments, python versions, so on. You might also need to install python dev files, so to ‘brute-force’ the installation I would run all of these:

sudo apt-get install python-dev python3-dev
sudo apt-get install libmysqlclient-dev
pip install MySQL-python
pip install pymysql
pip install mysqlclient

You should be good to go with the accepted answer. And can remove the unnecessary packages if that’s important to you.


回答 5

运行这些命令

sudo apt-get install python-dev python3-dev
sudo apt-get install libmysqlclient-dev
pip install MySQL-python 
pip install pymysql
pip install mysqlclient

然后像这样配置settings.py

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.mysql',
        'NAME': 'django_db',
        'HOST': '127.0.0.1',
        'PORT': '3306',
        'USER': 'root',
        'PASSWORD': '123456',
    }
}

享受mysql连接

Run these commands

sudo apt-get install python-dev python3-dev
sudo apt-get install libmysqlclient-dev
pip install MySQL-python 
pip install pymysql
pip install mysqlclient

Then configure settings.py like

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.mysql',
        'NAME': 'django_db',
        'HOST': '127.0.0.1',
        'PORT': '3306',
        'USER': 'root',
        'PASSWORD': '123456',
    }
}

Enjoy mysql connection


回答 6

安迪的答案很有帮助,但是如果您担心要在django设置中公开数据库密码,我建议在mysql连接上遵循django官方配置:https : //docs.djangoproject.com/en/1.7/ref/databases/

在这里引用为:

# settings.py
DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.mysql',
        'OPTIONS': {
            'read_default_file': '/path/to/my.cnf',
        },
    }
}


# my.cnf
[client]
database = NAME
user = USER
password = PASSWORD
default-character-set = utf8

要在设置中替换“ HOST”:“ 127.0.0.1”,只需将其添加到my.cnf中:

# my.cnf
[client]
database = NAME
host = HOST NAME or IP
user = USER
password = PASSWORD
default-character-set = utf8

另一个有用的选项是为django设置存储引擎,您可能需要在setting.py中使用它:

'OPTIONS': {
   'init_command': 'SET storage_engine=INNODB',
}

Andy’s answer helps but if you have concern on exposing your database password in your django setting, I suggest to follow django official configuration on mysql connection: https://docs.djangoproject.com/en/1.7/ref/databases/

Quoted here as:

# settings.py
DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.mysql',
        'OPTIONS': {
            'read_default_file': '/path/to/my.cnf',
        },
    }
}


# my.cnf
[client]
database = NAME
user = USER
password = PASSWORD
default-character-set = utf8

To replace ‘HOST’: ‘127.0.0.1’ in setting, simply add it in my.cnf:

# my.cnf
[client]
database = NAME
host = HOST NAME or IP
user = USER
password = PASSWORD
default-character-set = utf8

Another OPTION that is useful, is to set your storage engine for django, you might want it in your setting.py:

'OPTIONS': {
   'init_command': 'SET storage_engine=INNODB',
}

回答 7

settings.py

DATABASES = {
'default': {
    'ENGINE': 'django.db.backends.mysql',
    'NAME': 'django',
    'USER': 'root',
    'PASSWORD': '*****',
    'HOST': '***.***.***.***',
    'PORT': '3306',
    'OPTIONS': {
        'autocommit': True,
    },
}

}

然后:

python manage.py migrate

如果成功将生成这些表:

auth_group
auth_group_permissions
auth_permission
auth_user
auth_user_groups
auth_user_user_permissions
django_admin_log
django_content_type
django_migrations
django_session

您将可以使用mysql。

这是一个展示示例,请在Django 1.11.5版上进行测试: Django-pool-showcase

settings.py

DATABASES = {
'default': {
    'ENGINE': 'django.db.backends.mysql',
    'NAME': 'django',
    'USER': 'root',
    'PASSWORD': '*****',
    'HOST': '***.***.***.***',
    'PORT': '3306',
    'OPTIONS': {
        'autocommit': True,
    },
}

}

then:

python manage.py migrate

if success will generate theses tables:

auth_group
auth_group_permissions
auth_permission
auth_user
auth_user_groups
auth_user_user_permissions
django_admin_log
django_content_type
django_migrations
django_session

and u will can use mysql.

this is a showcase example ,test on Django version 1.11.5: Django-pool-showcase


回答 8

  1. 安装 mysqlclient

sudo pip3 install mysqlclient

如果出现错误:

命令“ python setup.py egg_info”在/ tmp / pip-install-dbljg4tx / mysqlclient /中失败,错误代码为1

然后:

 1. sudo apt install libmysqlclient-dev python-mysqldb

 2. sudo pip3 install mysqlclient

  1. 修改settings.py

    DATABASES = {
        'default': {
            'ENGINE': 'django.db.backends.mysql',
            'NAME': 'website',
            'USER': 'root',
            'PASSWORD': '',
            'HOST': '127.0.0.1',
            'PORT': '3306',
            'OPTION': {'init_command':"SET sql_mode='STRICT_TRANS_TABLE',"},
        }
    }
  1. Install mysqlclient

sudo pip3 install mysqlclient

if you get error:

Command “python setup.py egg_info” failed with error code 1 in /tmp/pip-install-dbljg4tx/mysqlclient/

then:

 1. sudo apt install libmysqlclient-dev python-mysqldb

 2. sudo pip3 install mysqlclient

  1. Modify settings.py

    DATABASES = {
        'default': {
            'ENGINE': 'django.db.backends.mysql',
            'NAME': 'website',
            'USER': 'root',
            'PASSWORD': '',
            'HOST': '127.0.0.1',
            'PORT': '3306',
            'OPTION': {'init_command':"SET sql_mode='STRICT_TRANS_TABLE',"},
        }
    }
    

回答 9

请按照给定的步骤进行设置以使用MySQL数据库:

1) Install MySQL Database Connector :

    sudo apt-get install libmysqlclient-dev

2) Install the mysqlclient library :

    pip install mysqlclient

3) Install MySQL server, with the following command :

    sudo apt-get install mysql-server

4) Create the Database :

    i) Verify that the MySQL service is running:

        systemctl status mysql.service

    ii) Log in with your MySQL credentials using the following command where -u is the flag for declaring your username and -p is the flag that tells MySQL that this user requires a password :  

        mysql -u db_user -p


    iii) CREATE DATABASE db_name;

    iv) Exit MySQL server, press CTRL + D.

5) Add the MySQL Database Connection to your Application:

    i) Navigate to the settings.py file and replace the current DATABASES lines with the following:

        # Database
        # https://docs.djangoproject.com/en/2.0/ref/settings/#databases

        DATABASES = {
            'default': {
                'ENGINE': 'django.db.backends.mysql',
                'OPTIONS': {
                    'read_default_file': '/etc/mysql/my.cnf',
                },
            }
        }
        ...

    ii) Next, lets edit the config file so that it has your MySQL credentials. Use vi as sudo to edit the file and add the following information:

        sudo vi /etc/mysql/my.cnf

        database = db_name
        user = db_user
        password = db_password
        default-character-set = utf8

6) Once the file has been edited, we need to restart MySQL for the changes to take effect :

    systemctl daemon-reload

    systemctl restart mysql

7) Test MySQL Connection to Application:

    python manage.py runserver your-server-ip:8000

Follow the given steps in order to setup it up to use MySQL database:

1) Install MySQL Database Connector :

    sudo apt-get install libmysqlclient-dev

2) Install the mysqlclient library :

    pip install mysqlclient

3) Install MySQL server, with the following command :

    sudo apt-get install mysql-server

4) Create the Database :

    i) Verify that the MySQL service is running:

        systemctl status mysql.service

    ii) Log in with your MySQL credentials using the following command where -u is the flag for declaring your username and -p is the flag that tells MySQL that this user requires a password :  

        mysql -u db_user -p


    iii) CREATE DATABASE db_name;

    iv) Exit MySQL server, press CTRL + D.

5) Add the MySQL Database Connection to your Application:

    i) Navigate to the settings.py file and replace the current DATABASES lines with the following:

        # Database
        # https://docs.djangoproject.com/en/2.0/ref/settings/#databases

        DATABASES = {
            'default': {
                'ENGINE': 'django.db.backends.mysql',
                'OPTIONS': {
                    'read_default_file': '/etc/mysql/my.cnf',
                },
            }
        }
        ...

    ii) Next, let’s edit the config file so that it has your MySQL credentials. Use vi as sudo to edit the file and add the following information:

        sudo vi /etc/mysql/my.cnf

        database = db_name
        user = db_user
        password = db_password
        default-character-set = utf8

6) Once the file has been edited, we need to restart MySQL for the changes to take effect :

    systemctl daemon-reload

    systemctl restart mysql

7) Test MySQL Connection to Application:

    python manage.py runserver your-server-ip:8000

回答 10

您必须首先创建一个MySQL数据库。然后转到settings.py文件并'DATABASES'使用您的MySQL凭据编辑字典:

DATABASES = {
 'default': {
 'ENGINE': 'django.db.backends.mysql',
 'NAME': 'YOUR_DATABASE_NAME',
 'USER': 'YOUR_MYSQL_USER',
 'PASSWORD': 'YOUR_MYSQL_PASS',
 'HOST': 'localhost',   # Or an IP that your DB is hosted on
 'PORT': '3306',
 }
}

这是用于将Django设置为在virtualenv上使用MySQL的完整安装指南:

http://codex.themedelta.com/how-to-install-django-with-mysql-in-a-virtualenv-on-linux/

You must create a MySQL database first. Then go to settings.py file and edit the 'DATABASES' dictionary with your MySQL credentials:

DATABASES = {
 'default': {
 'ENGINE': 'django.db.backends.mysql',
 'NAME': 'YOUR_DATABASE_NAME',
 'USER': 'YOUR_MYSQL_USER',
 'PASSWORD': 'YOUR_MYSQL_PASS',
 'HOST': 'localhost',   # Or an IP that your DB is hosted on
 'PORT': '3306',
 }
}

Here is a complete installation guide for setting up Django to use MySQL on a virtualenv:

http://codex.themedelta.com/how-to-install-django-with-mysql-in-a-virtualenv-on-linux/


如何在两个Django应用之间移动模型(Django 1.7)

问题:如何在两个Django应用之间移动模型(Django 1.7)

因此,大约一年前,我开始了一个项目,像所有新开发人员一样,我并没有真正专注于结构,但是现在我与Django一起走得更远,它开始似乎表明我的项目布局主要是我的模型在结构上很糟糕。

我的模型主要保存在单个应用程序中,实际上这些模型中的大多数应该放在自己的单个应用程序中,我确实尝试解决了此问题并将其向南移动,但是由于外键等原因,我发现它很棘手,而且确实很困难。

但是,由于Django 1.7并内置了对迁移的支持,现在有更好的方法吗?

So about a year ago I started a project and like all new developers I didn’t really focus too much on the structure, however now I am further along with Django it has started to appear that my project layout mainly my models are horrible in structure.

I have models mainly held in a single app and really most of these models should be in their own individual apps, I did try and resolve this and move them with south however I found it tricky and really difficult due to foreign keys ect.

However due to Django 1.7 and built in support for migrations is there a better way to do this now?


回答 0

我正在删除旧答案,因为这可能会导致数据丢失。如ozan所述,我们可以在每个应用中创建2个迁移。这篇文章下面的评论指的是我的旧答案。

第一次迁移以从第一个应用中删除模型。

$ python manage.py makemigrations old_app --empty

编辑迁移文件以包括这些操作。

class Migration(migrations.Migration):

    database_operations = [migrations.AlterModelTable('TheModel', 'newapp_themodel')]

    state_operations = [migrations.DeleteModel('TheModel')]

    operations = [
      migrations.SeparateDatabaseAndState(
        database_operations=database_operations,
        state_operations=state_operations)
    ]

第二次迁移取决于第一次迁移并在第二个应用程序中创建新表。将模型代码移至第二个应用程序后

$ python manage.py makemigrations new_app 

然后将迁移文件编辑为类似的内容。

class Migration(migrations.Migration):

    dependencies = [
        ('old_app', 'above_migration')
    ]

    state_operations = [
        migrations.CreateModel(
            name='TheModel',
            fields=[
                ('id', models.AutoField(verbose_name='ID', serialize=False, auto_created=True, primary_key=True)),
            ],
            options={
                'db_table': 'newapp_themodel',
            },
            bases=(models.Model,),
        )
    ]

    operations = [
        migrations.SeparateDatabaseAndState(state_operations=state_operations)
    ]

I am removing the old answer as may result in data loss. As ozan mentioned, we can create 2 migrations one in each app. The comments below this post refer to my old answer.

First migration to remove model from 1st app.

$ python manage.py makemigrations old_app --empty

Edit migration file to include these operations.

class Migration(migrations.Migration):

    database_operations = [migrations.AlterModelTable('TheModel', 'newapp_themodel')]

    state_operations = [migrations.DeleteModel('TheModel')]

    operations = [
      migrations.SeparateDatabaseAndState(
        database_operations=database_operations,
        state_operations=state_operations)
    ]

Second migration which depends on first migration and create the new table in 2nd app. After moving model code to 2nd app

$ python manage.py makemigrations new_app 

and edit migration file to something like this.

class Migration(migrations.Migration):

    dependencies = [
        ('old_app', 'above_migration')
    ]

    state_operations = [
        migrations.CreateModel(
            name='TheModel',
            fields=[
                ('id', models.AutoField(verbose_name='ID', serialize=False, auto_created=True, primary_key=True)),
            ],
            options={
                'db_table': 'newapp_themodel',
            },
            bases=(models.Model,),
        )
    ]

    operations = [
        migrations.SeparateDatabaseAndState(state_operations=state_operations)
    ]

回答 1

使用可以很容易地做到这一点migrations.SeparateDatabaseAndState。基本上,我们使用数据库操作来同时重命名表,同时使用两个状态操作从一个应用程序的历史记录中删除模型并在另一个应用程序的历史记录中创建模型。

从旧应用中删除

python manage.py makemigrations old_app --empty

在迁移中:

class Migration(migrations.Migration):

    dependencies = []

    database_operations = [
        migrations.AlterModelTable('TheModel', 'newapp_themodel')
    ]

    state_operations = [
        migrations.DeleteModel('TheModel')
    ]

    operations = [
        migrations.SeparateDatabaseAndState(
            database_operations=database_operations,
            state_operations=state_operations)
    ]

添加到新应用

首先,将模型复制到新应用的model.py中,然后:

python manage.py makemigrations new_app

这将生成一个迁移CreateModel操作,其中天真的操作是唯一的操作。将其包装在一个SeparateDatabaseAndState操作中,这样我们就不会尝试重新创建表。还包括先前的迁移作为依赖项:

class Migration(migrations.Migration):

    dependencies = [
        ('old_app', 'above_migration')
    ]

    state_operations = [
        migrations.CreateModel(
            name='TheModel',
            fields=[
                ('id', models.AutoField(verbose_name='ID', serialize=False, auto_created=True, primary_key=True)),
            ],
            options={
                'db_table': 'newapp_themodel',
            },
            bases=(models.Model,),
        )
    ]

    operations = [
        migrations.SeparateDatabaseAndState(state_operations=state_operations)
    ]

This can be done fairly easily using migrations.SeparateDatabaseAndState. Basically, we use a database operation to rename the table concurrently with two state operations to remove the model from one app’s history and create it in another’s.

Remove from old app

python manage.py makemigrations old_app --empty

In the migration:

class Migration(migrations.Migration):

    dependencies = []

    database_operations = [
        migrations.AlterModelTable('TheModel', 'newapp_themodel')
    ]

    state_operations = [
        migrations.DeleteModel('TheModel')
    ]

    operations = [
        migrations.SeparateDatabaseAndState(
            database_operations=database_operations,
            state_operations=state_operations)
    ]

Add to new app

First, copy the model to the new app’s model.py, then:

python manage.py makemigrations new_app

This will generate a migration with a naive CreateModel operation as the sole operation. Wrap that in a SeparateDatabaseAndState operation such that we don’t try to recreate the table. Also include the prior migration as a dependency:

class Migration(migrations.Migration):

    dependencies = [
        ('old_app', 'above_migration')
    ]

    state_operations = [
        migrations.CreateModel(
            name='TheModel',
            fields=[
                ('id', models.AutoField(verbose_name='ID', serialize=False, auto_created=True, primary_key=True)),
            ],
            options={
                'db_table': 'newapp_themodel',
            },
            bases=(models.Model,),
        )
    ]

    operations = [
        migrations.SeparateDatabaseAndState(state_operations=state_operations)
    ]

回答 2

我遇到了同样的问题。 奥赞的回答对我有很大帮助,但不幸的是还不够。确实,我有几个ForeignKey链接到我想移动的模型。经过一番头痛之后,我发现了解决方案,因此决定发布该解决方案以解决人们的时间问题。

您还需要2个步骤:

  1. 在执行任何操作之前,请将所有ForeignKey链接更改TheModelIntegerfield。然后跑python manage.py makemigrations
  2. 完成Ozan的步骤后,请重新转换外键:放回ForeignKey(TheModel)而不是IntegerField()。然后再次进行迁移(python manage.py makemigrations)。然后,您可以迁移,它应该可以工作(python manage.py migrate

希望能帮助到你。当然,在尝试生产之前,请先在本地进行测试,以避免出现意外情况:)

I encountered the same problem. Ozan’s answer helped me a lot but unfortunately was not enough. Indeed I had several ForeignKey linking to the model I wanted to move. After some headache I found the solution so decided to post it to solve people time.

You need 2 more steps:

  1. Before doing anything, change all your ForeignKey linking to TheModel into Integerfield. Then run python manage.py makemigrations
  2. After doing Ozan’s steps, re-convert your foreign keys: put back ForeignKey(TheModel)instead of IntegerField(). Then make the migrations again (python manage.py makemigrations). You can then migrate and it should work (python manage.py migrate)

Hope it helps. Of course test it in local before trying in production to avoid bad suprises :)


回答 3

我是如何做到的(在PostgreSQL == 1.8上进行了测试,使用postgres,可能也是1.7)

情况

app1.YourModel

但您希望它转到: app2.YourModel

  1. 将YourModel(代码)从app1复制到app2。
  2. 将此添加到app2.YourModel:

    Class Meta:
        db_table = 'app1_yourmodel'
  3. $ python manage.py makemigrations app2

  4. 使用migrations.CreateModel()语句在app2中进行了新的迁移(例如0009_auto_something.py),将该语句移至app2的初始迁移(例如0001_initial.py)(就像它一直在那儿一样)。现在删除创建的迁移= 0009_auto_something.py

  5. 就像您执行操作一样,就像app2.YourModel一直存在一样,现在从迁移中删除app1.YourModel的存在。含义:注释掉CreateModel语句,以及之后使用的所有调整或数据迁移。

  6. 当然,必须在您的项目中将对app1.YourModel的每个引用都更改为app2.YourModel。另外,不要忘记迁移中所有可能的app1.YourModel外键都必须更改为app2.YourModel。

  7. 现在,如果您执行$ python manage.py迁移,则什么都没有改变,同样,当您执行$ python manage.py makemigrations时,也不会检测到新的东西。

  8. 现在画龙点睛:从app2.YourModel中删除Class Meta并执行$ python manage.py makemigrations app2 && python manage.py migration app2(如果您研究此迁移,您将看到类似以下的内容:)

        migrations.AlterModelTable(
        name='yourmodel',
        table=None,
    ),

table = None,表示它将采用默认的表名,在这种情况下为app2_yourmodel。

  1. 完成,保存数据。

PS在迁移期间,它将看到content_type app1.yourmodel已被删除并且可以删除。您可以对此说是,但前提是您不使用它。如果您非常依赖它来使该内容类型的FK保持完整,则不要回答yes或no,而是手动进入该时间的db,并删除co​​ntentype app2.yourmodel,然后重命名contenttype app1。 yourmodel到app2.yourmodel,然后继续回答否。

How I did it (tested on Django==1.8, with postgres, so probably also 1.7)

Situation

app1.YourModel

but you want it to go to: app2.YourModel

  1. Copy YourModel (the code) from app1 to app2.
  2. add this to app2.YourModel:

    Class Meta:
        db_table = 'app1_yourmodel'
    
  3. $ python manage.py makemigrations app2

  4. A new migration (e.g. 0009_auto_something.py) is made in app2 with a migrations.CreateModel() statement, move this statement to the initial migration of app2 (e.g. 0001_initial.py) (it will be just like it always have been there). And now remove the created migration = 0009_auto_something.py

  5. Just as you act, like app2.YourModel always has been there, now remove the existence of app1.YourModel from your migrations. Meaning: comment out the CreateModel statements, and every adjustment or datamigration you used after that.

  6. And of course, every reference to app1.YourModel has to be changed to app2.YourModel through your project. Also, don’t forget that all possible foreign keys to app1.YourModel in migrations have to be changed to app2.YourModel

  7. Now if you do $ python manage.py migrate, nothing has changed, also when you do $ python manage.py makemigrations, nothing new has been detected.

  8. Now the finishing touch: remove the Class Meta from app2.YourModel and do $ python manage.py makemigrations app2 && python manage.py migrate app2 (if you look into this migration you’ll see something like this:)

        migrations.AlterModelTable(
        name='yourmodel',
        table=None,
    ),
    

table=None, means it will take the default table-name, which in this case will be app2_yourmodel.

  1. DONE, with data saved.

P.S during the migration it will see that that content_type app1.yourmodel has been removed and can be deleted. You can say yes to that but only if you don’t use it. In case you heavily depend on it to have FKs to that content-type be intact, don’t answer yes or no yet, but go into the db that time manually, and remove the contentype app2.yourmodel, and rename the contenttype app1.yourmodel to app2.yourmodel, and then continue by answering no.


回答 4

我会感到紧张的手工编码迁移(这是Ozan的回答所要求),因此以下内容结合了Ozan和Michael的策略以最大程度地减少所需的手工编码量:

  1. 在移动任何模型之前,请通过运行确保使用干净的基线makemigrations
  2. 将模型的代码从app1移至app2
  3. 按照@Michael的建议,我们使用db_table“新”模型上的Meta选项将新模型指向旧数据库表:

    class Meta:
        db_table = 'app1_yourmodel'
  4. 运行makemigrations。这将CreateModelapp2DeleteModel中生成app1。从技术上讲,这些迁移将引用完全相同的表,并且将删除(包括所有数据)并重新创建表。

  5. 实际上,我们不希望(或不需要)对表做任何事情。我们只需要Django相信所做的更改即可。根据@Ozan的回答,中的state_operations标志将SeparateDatabaseAndState执行此操作。因此,我们使用来将所有migrations条目包装在两个迁移文件SeparateDatabaseAndState(state_operations=[...])。例如,

    operations = [
        ...
        migrations.DeleteModel(
            name='YourModel',
        ),
        ...
    ]

    变成

    operations = [
        migrations.SeparateDatabaseAndState(state_operations=[
            ...
            migrations.DeleteModel(
                name='YourModel',
            ),
            ...
        ])
    ]
  6. 您还需要确保新的“虚拟” CreateModel迁移取决于实际创建或更改原始表的任何迁移。例如,如果您的新迁移是app2.migrations.0004_auto_<date>(针对Create)和app1.migrations.0007_auto_<date>(针对Delete),则最简单的操作是:

    • 打开app1.migrations.0007_auto_<date>并复制其app1依赖项(例如 ('app1', '0006...'),)。这是“立即优先”的迁移app1,应该包括对所有实际模型构建逻辑的依赖。
    • 打开 app2.migrations.0004_auto_<date>并将刚刚复制的依赖项添加到其dependencies列表中。

如果您ForeignKey与要移动的模型有关系,则上述方法可能无效。发生这种情况是因为:

  • 不会自动为ForeignKey更改创建依赖关系
  • 我们不想包装这些ForeignKey更改,state_operations因此我们需要确保它们与表操作是分开的。

注意:Django 2.2添加了一个警告(models.E028),它破坏了此方法。您也许可以解决该问题,managed=False但我尚未对其进行测试。

“最小”操作集因情况而异,但是以下过程应适用于大多数/所有ForeignKey迁移:

  1. app1复制模型app2,设置db_table,但不要更改任何FK引用。
  2. 运行makemigrations并包装所有app2迁移state_operations(请参见上文)
    • 如上所述,将依赖项添加app2 CreateTable到最新的app1迁移中
  3. 将所有FK参考指向新模型。如果您不使用字符串引用,请将旧模型移到models.py(不要删除它)的底部,这样它就不会与导入的类竞争。
  4. 运行,makemigrations但不要包装任何内容state_operations(实际上应该发生FK更改)。将所有ForeignKey迁移(即AlterField)中的依赖项添加到其中的CreateTable迁移中app2(下一步将需要此列表,因此请对其进行跟踪)。例如:

    • 找到包含迁移CreateModelapp2.migrations.0002_auto_<date>和复制迁移的名称。
    • 查找所有具有该模型的ForeignKey的迁移(例如,通过搜索app2.YourModel找到类似以下的迁移:

      class Migration(migrations.Migration):
      
          dependencies = [
              ('otherapp', '0001_initial'),
          ]
      
          operations = [
              migrations.AlterField(
                  model_name='relatedmodel',
                  name='fieldname',
                  field=models.ForeignKey(... to='app2.YourModel'),
              ),
          ]
    • CreateModel迁移作为依赖项添加:

      class Migration(migrations.Migration):
      
          dependencies = [
              ('otherapp', '0001_initial'),
              ('app2', '0002_auto_<date>'),
          ]  
  5. 从中删除模型 app1

  6. 运行迁移makemigrations并将其包装在app1state_operations
    • 为上一步的所有ForeignKey迁移(即AlterField)添加一个依赖项(可能包括app1和中的迁移app2)。
    • 当我构建这些迁移时,DeleteTable已经依赖于AlterField迁移了,因此我不需要手动执行它(即Alterbefore Delete)。

在这一点上,Django是个不错的选择。新模型指向旧表,并且Django的迁移使它确信所有内容都已适当地重新放置。最大的警告(来自@Michael的答案)是ContentType为新模型创建了一个新模型。如果您链接(例如通过ForeignKey)到内容类型,则需要创建一个迁移来更新ContentType表。

我想自己清理一下(元选项和表名),所以我使用了以下过程(来自@Michael):

  1. 删除db_table元条目
  2. makemigrations再次运行以生成数据库重命名
  3. 编辑此最后的迁移,并确保它取决于DeleteTable迁移。似乎没有必要,因为它应该Delete纯粹是逻辑上的,但是app1_yourmodel如果我不这样做,就会遇到错误(例如,不存在)。

I get nervous hand-coding migrations (as is required by Ozan’s answer) so the following combines Ozan’s and Michael’s strategies to minimize the amount of hand-coding required:

  1. Before moving any models, make sure you’re working with a clean baseline by running makemigrations.
  2. Move the code for the Model from app1 to app2
  3. As recommended by @Michael, we point the new model to the old database table using the db_table Meta option on the “new” model:

    class Meta:
        db_table = 'app1_yourmodel'
    
  4. Run makemigrations. This will generate CreateModel in app2 and DeleteModel in app1. Technically, these migrations refer to the exact same table and would remove (including all data) and re-create the table.

  5. In reality, we don’t want (or need) to do anything to the table. We just need Django to believe that the change has been made. Per @Ozan’s answer, the state_operations flag in SeparateDatabaseAndState does this. So we wrap all of the migrations entries IN BOTH MIGRATIONS FILES with SeparateDatabaseAndState(state_operations=[...]). For example,

    operations = [
        ...
        migrations.DeleteModel(
            name='YourModel',
        ),
        ...
    ]
    

    becomes

    operations = [
        migrations.SeparateDatabaseAndState(state_operations=[
            ...
            migrations.DeleteModel(
                name='YourModel',
            ),
            ...
        ])
    ]
    
  6. You also need to make sure the new “virtual” CreateModel migration depends on any migration that actually created or altered the original table. For example, if your new migrations are app2.migrations.0004_auto_<date> (for the Create) and app1.migrations.0007_auto_<date> (for the Delete), the simplest thing to do is:

    • Open app1.migrations.0007_auto_<date> and copy its app1 dependency (e.g. ('app1', '0006...'),). This is the “immediately prior” migration in app1 and should include dependencies on all of the actual model building logic.
    • Open app2.migrations.0004_auto_<date> and add the dependency you just copied to its dependencies list.

If you have ForeignKey relationship(s) to the model you’re moving, the above may not work. This happens because:

  • Dependencies are not automatically created for the ForeignKey changes
  • We do not want to wrap the ForeignKey changes in state_operations so we need to ensure they are separate from the table operations.

NOTE: Django 2.2 added a warning (models.E028) that breaks this method. You may be able to work around it with managed=False but I have not tested it.

The “minimum” set of operations differ depending on the situation, but the following procedure should work for most/all ForeignKey migrations:

  1. COPY the model from app1 to app2, set db_table, but DON’T change any FK references.
  2. Run makemigrations and wrap all app2 migration in state_operations (see above)
    • As above, add a dependency in the app2 CreateTable to the latest app1 migration
  3. Point all of the FK references to the new model. If you aren’t using string references, move the old model to the bottom of models.py (DON’T remove it) so it doesn’t compete with the imported class.
  4. Run makemigrations but DON’T wrap anything in state_operations (the FK changes should actually happen). Add a dependency in all the ForeignKey migrations (i.e. AlterField) to the CreateTable migration in app2 (you’ll need this list for the next step so keep track of them). For example:

    • Find the migration that includes the CreateModel e.g. app2.migrations.0002_auto_<date> and copy the name of that migration.
    • Find all migrations that have a ForeignKey to that model (e.g. by searching app2.YourModel to find migrations like:

      class Migration(migrations.Migration):
      
          dependencies = [
              ('otherapp', '0001_initial'),
          ]
      
          operations = [
              migrations.AlterField(
                  model_name='relatedmodel',
                  name='fieldname',
                  field=models.ForeignKey(... to='app2.YourModel'),
              ),
          ]
      
    • Add the CreateModel migration as as a dependency:

      class Migration(migrations.Migration):
      
          dependencies = [
              ('otherapp', '0001_initial'),
              ('app2', '0002_auto_<date>'),
          ]  
      
  5. Remove the models from app1

  6. Run makemigrations and wrap the app1 migration in state_operations.
    • Add a dependency to all of the ForeignKey migrations (i.e. AlterField) from the previous step (may include migrations in app1 and app2).
    • When I built these migrations, the DeleteTable already depended on the AlterField migrations so I didn’t need to manually enforce it (i.e. Alter before Delete).

At this point, Django is good to go. The new model points to the old table and Django’s migrations have convinced it that everything has been relocated appropriately. The big caveat (from @Michael’s answer) is that a new ContentType is created for the new model. If you link (e.g. by ForeignKey) to content types, you’ll need to create a migration to update the ContentType table.

I wanted to cleanup after myself (Meta options and table names) so I used the following procedure (from @Michael):

  1. Remove the db_table Meta entry
  2. Run makemigrations again to generate the database rename
  3. Edit this last migration and make sure it depends on the DeleteTable migration. It doesn’t seem like it should be necessary as the Delete should be purely logical, but I’ve run into errors (e.g. app1_yourmodel doesn’t exist) if I don’t.

回答 5

如果数据不是很大或太复杂,但仍然很重要,则另一种骇人听闻的选择是:

  • 使用manage.py dumpdata获取数据固定装置
  • 继续正确建模更改和迁移,而不涉及更改
  • 全局将设备从旧型号和应用名称替换为新的
  • 使用manage.py loaddata加载数据

Another hacky alternative if the data is not big or too complicated, but still important to maintain, is to:

  • Get data fixtures using manage.py dumpdata
  • Proceed to model changes and migrations properly, without relating the changes
  • Global replace the fixtures from the old model and app names to the new
  • Load data using manage.py loaddata

回答 6

从我在https://stackoverflow.com/a/47392970/8971048的回答中复制

万一您需要移动模型而又无权访问该应用程序(或者您不想访问该应用程序),则可以创建一个新的Operation并考虑仅在迁移的模型不具有该模型的情况下创建一个新模型存在。

在此示例中,我将“ MyModel”从old_app传递到myapp。

class MigrateOrCreateTable(migrations.CreateModel):
    def __init__(self, source_table, dst_table, *args, **kwargs):
        super(MigrateOrCreateTable, self).__init__(*args, **kwargs)
        self.source_table = source_table
        self.dst_table = dst_table

    def database_forwards(self, app_label, schema_editor, from_state, to_state):
        table_exists = self.source_table in schema_editor.connection.introspection.table_names()
        if table_exists:
            with schema_editor.connection.cursor() as cursor:
                cursor.execute("RENAME TABLE {} TO {};".format(self.source_table, self.dst_table))
        else:
            return super(MigrateOrCreateTable, self).database_forwards(app_label, schema_editor, from_state, to_state)


class Migration(migrations.Migration):

    dependencies = [
        ('myapp', '0002_some_migration'),
    ]

    operations = [
        MigrateOrCreateTable(
            source_table='old_app_mymodel',
            dst_table='myapp_mymodel',
            name='MyModel',
            fields=[
                ('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
                ('name', models.CharField(max_length=18))
            ],
        ),
    ]

Copied from my answer at https://stackoverflow.com/a/47392970/8971048

In case you need to move the model and you don’t have access to the app anymore (or you don’t want the access), you can create a new Operation and consider to create a new model only if the migrated model does not exist.

In this example I am passing ‘MyModel’ from old_app to myapp.

class MigrateOrCreateTable(migrations.CreateModel):
    def __init__(self, source_table, dst_table, *args, **kwargs):
        super(MigrateOrCreateTable, self).__init__(*args, **kwargs)
        self.source_table = source_table
        self.dst_table = dst_table

    def database_forwards(self, app_label, schema_editor, from_state, to_state):
        table_exists = self.source_table in schema_editor.connection.introspection.table_names()
        if table_exists:
            with schema_editor.connection.cursor() as cursor:
                cursor.execute("RENAME TABLE {} TO {};".format(self.source_table, self.dst_table))
        else:
            return super(MigrateOrCreateTable, self).database_forwards(app_label, schema_editor, from_state, to_state)


class Migration(migrations.Migration):

    dependencies = [
        ('myapp', '0002_some_migration'),
    ]

    operations = [
        MigrateOrCreateTable(
            source_table='old_app_mymodel',
            dst_table='myapp_mymodel',
            name='MyModel',
            fields=[
                ('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
                ('name', models.CharField(max_length=18))
            ],
        ),
    ]

回答 7

这是经过粗略测试的,所以不要忘记备份数据库!!!

例如,有两个应用程序:src_appdst_app,我们希望将模型MoveMesrc_app移至dst_app

为两个应用程序创建空迁移:

python manage.py makemigrations --empty src_app
python manage.py makemigrations --empty dst_app

让我们假设,新的迁移是XXX1_src_app_newXXX1_dst_app_new,之前的主要迁移是XXX0_src_app_oldXXX0_dst_app_old

添加一个操作,该操作重命名MoveMe模型的表,并将其在ProjectState中的app_label重命名为XXX1_dst_app_new。不要忘记增加对XXX0_src_app_old迁移的依赖。产生的XXX1_dst_app_new迁移是:

# -*- coding: utf-8 -*-
from __future__ import unicode_literals

from django.db import models, migrations

# this operations is almost the same as RenameModel
# https://github.com/django/django/blob/1.7/django/db/migrations/operations/models.py#L104
class MoveModelFromOtherApp(migrations.operations.base.Operation):

    def __init__(self, name, old_app_label):
        self.name = name
        self.old_app_label = old_app_label

    def state_forwards(self, app_label, state):

        # Get all of the related objects we need to repoint
        apps = state.render(skip_cache=True)
        model = apps.get_model(self.old_app_label, self.name)
        related_objects = model._meta.get_all_related_objects()
        related_m2m_objects = model._meta.get_all_related_many_to_many_objects()
        # Rename the model
        state.models[app_label, self.name.lower()] = state.models.pop(
            (self.old_app_label, self.name.lower())
        )
        state.models[app_label, self.name.lower()].app_label = app_label
        for model_state in state.models.values():
            try:
                i = model_state.bases.index("%s.%s" % (self.old_app_label, self.name.lower()))
                model_state.bases = model_state.bases[:i] + ("%s.%s" % (app_label, self.name.lower()),) + model_state.bases[i+1:]
            except ValueError:
                pass
        # Repoint the FKs and M2Ms pointing to us
        for related_object in (related_objects + related_m2m_objects):
            # Use the new related key for self referential related objects.
            if related_object.model == model:
                related_key = (app_label, self.name.lower())
            else:
                related_key = (
                    related_object.model._meta.app_label,
                    related_object.model._meta.object_name.lower(),
                )
            new_fields = []
            for name, field in state.models[related_key].fields:
                if name == related_object.field.name:
                    field = field.clone()
                    field.rel.to = "%s.%s" % (app_label, self.name)
                new_fields.append((name, field))
            state.models[related_key].fields = new_fields

    def database_forwards(self, app_label, schema_editor, from_state, to_state):
        old_apps = from_state.render()
        new_apps = to_state.render()
        old_model = old_apps.get_model(self.old_app_label, self.name)
        new_model = new_apps.get_model(app_label, self.name)
        if self.allowed_to_migrate(schema_editor.connection.alias, new_model):
            # Move the main table
            schema_editor.alter_db_table(
                new_model,
                old_model._meta.db_table,
                new_model._meta.db_table,
            )
            # Alter the fields pointing to us
            related_objects = old_model._meta.get_all_related_objects()
            related_m2m_objects = old_model._meta.get_all_related_many_to_many_objects()
            for related_object in (related_objects + related_m2m_objects):
                if related_object.model == old_model:
                    model = new_model
                    related_key = (app_label, self.name.lower())
                else:
                    model = related_object.model
                    related_key = (
                        related_object.model._meta.app_label,
                        related_object.model._meta.object_name.lower(),
                    )
                to_field = new_apps.get_model(
                    *related_key
                )._meta.get_field_by_name(related_object.field.name)[0]
                schema_editor.alter_field(
                    model,
                    related_object.field,
                    to_field,
                )

    def database_backwards(self, app_label, schema_editor, from_state, to_state):
        self.old_app_label, app_label = app_label, self.old_app_label
        self.database_forwards(app_label, schema_editor, from_state, to_state)
        app_label, self.old_app_label = self.old_app_label, app_label

    def describe(self):
        return "Move %s from %s" % (self.name, self.old_app_label)


class Migration(migrations.Migration):

    dependencies = [
       ('dst_app', 'XXX0_dst_app_old'),
       ('src_app', 'XXX0_src_app_old'),
    ]

    operations = [
        MoveModelFromOtherApp('MoveMe', 'src_app'),
    ]

将依赖项添加XXX1_dst_app_newXXX1_src_app_newXXX1_src_app_new是No-op迁移,可确保src_app在之后执行将来的迁移XXX1_dst_app_new

移动MoveMesrc_app/models.pydst_app/models.py。然后运行:

python manage.py migrate

就这样!

This is tested roughly, so do not forget to backup your DB!!!

For example, there are two apps: src_app and dst_app, we want to move model MoveMe from src_app to dst_app.

Create empty migrations for both apps:

python manage.py makemigrations --empty src_app
python manage.py makemigrations --empty dst_app

Let’s assume, that new migrations are XXX1_src_app_new and XXX1_dst_app_new, previuos top migrations are XXX0_src_app_old and XXX0_dst_app_old.

Add an operation that renames table for MoveMe model and renames its app_label in ProjectState to XXX1_dst_app_new. Do not forget to add dependency on XXX0_src_app_old migration. The resulting XXX1_dst_app_new migration is:

# -*- coding: utf-8 -*-
from __future__ import unicode_literals

from django.db import models, migrations

# this operations is almost the same as RenameModel
# https://github.com/django/django/blob/1.7/django/db/migrations/operations/models.py#L104
class MoveModelFromOtherApp(migrations.operations.base.Operation):

    def __init__(self, name, old_app_label):
        self.name = name
        self.old_app_label = old_app_label

    def state_forwards(self, app_label, state):

        # Get all of the related objects we need to repoint
        apps = state.render(skip_cache=True)
        model = apps.get_model(self.old_app_label, self.name)
        related_objects = model._meta.get_all_related_objects()
        related_m2m_objects = model._meta.get_all_related_many_to_many_objects()
        # Rename the model
        state.models[app_label, self.name.lower()] = state.models.pop(
            (self.old_app_label, self.name.lower())
        )
        state.models[app_label, self.name.lower()].app_label = app_label
        for model_state in state.models.values():
            try:
                i = model_state.bases.index("%s.%s" % (self.old_app_label, self.name.lower()))
                model_state.bases = model_state.bases[:i] + ("%s.%s" % (app_label, self.name.lower()),) + model_state.bases[i+1:]
            except ValueError:
                pass
        # Repoint the FKs and M2Ms pointing to us
        for related_object in (related_objects + related_m2m_objects):
            # Use the new related key for self referential related objects.
            if related_object.model == model:
                related_key = (app_label, self.name.lower())
            else:
                related_key = (
                    related_object.model._meta.app_label,
                    related_object.model._meta.object_name.lower(),
                )
            new_fields = []
            for name, field in state.models[related_key].fields:
                if name == related_object.field.name:
                    field = field.clone()
                    field.rel.to = "%s.%s" % (app_label, self.name)
                new_fields.append((name, field))
            state.models[related_key].fields = new_fields

    def database_forwards(self, app_label, schema_editor, from_state, to_state):
        old_apps = from_state.render()
        new_apps = to_state.render()
        old_model = old_apps.get_model(self.old_app_label, self.name)
        new_model = new_apps.get_model(app_label, self.name)
        if self.allowed_to_migrate(schema_editor.connection.alias, new_model):
            # Move the main table
            schema_editor.alter_db_table(
                new_model,
                old_model._meta.db_table,
                new_model._meta.db_table,
            )
            # Alter the fields pointing to us
            related_objects = old_model._meta.get_all_related_objects()
            related_m2m_objects = old_model._meta.get_all_related_many_to_many_objects()
            for related_object in (related_objects + related_m2m_objects):
                if related_object.model == old_model:
                    model = new_model
                    related_key = (app_label, self.name.lower())
                else:
                    model = related_object.model
                    related_key = (
                        related_object.model._meta.app_label,
                        related_object.model._meta.object_name.lower(),
                    )
                to_field = new_apps.get_model(
                    *related_key
                )._meta.get_field_by_name(related_object.field.name)[0]
                schema_editor.alter_field(
                    model,
                    related_object.field,
                    to_field,
                )

    def database_backwards(self, app_label, schema_editor, from_state, to_state):
        self.old_app_label, app_label = app_label, self.old_app_label
        self.database_forwards(app_label, schema_editor, from_state, to_state)
        app_label, self.old_app_label = self.old_app_label, app_label

    def describe(self):
        return "Move %s from %s" % (self.name, self.old_app_label)


class Migration(migrations.Migration):

    dependencies = [
       ('dst_app', 'XXX0_dst_app_old'),
       ('src_app', 'XXX0_src_app_old'),
    ]

    operations = [
        MoveModelFromOtherApp('MoveMe', 'src_app'),
    ]

Add dependency on XXX1_dst_app_new to XXX1_src_app_new. XXX1_src_app_new is no-op migration that is needed to make sure that future src_app migrations will be executed after XXX1_dst_app_new.

Move MoveMe from src_app/models.py to dst_app/models.py. Then run:

python manage.py migrate

That’s all!


回答 8

您可以尝试以下(未试用):

  1. 将模型从src_app移至dest_app
  2. 迁移dest_app; 确保模式迁移依赖于最新的src_app迁移(https://docs.djangoproject.com/en/dev/topics/migrations/#migration-files
  3. 将数据迁移添加到dest_app,从中复制所有数据src_app
  4. 迁移src_app; 确保模式迁移依赖于-的最新(数据)迁移,dest_app即:步骤3的迁移

请注意,您将复制整个表格,而不是移动整个表格,但是那样,两个应用程序都不必触摸属于另一个应用程序的表,我认为这更重要。

You can try the following (untested):

  1. move the model from src_app to dest_app
  2. migrate dest_app; make sure the schema migration depends on the latest src_app migration (https://docs.djangoproject.com/en/dev/topics/migrations/#migration-files)
  3. add a data migration to dest_app, that copies all data from src_app
  4. migrate src_app; make sure the schema migration depends on the latest (data) migration of dest_app — that is: the migration of step 3

Note that you will be copying the whole table, instead of moving it, but that way both apps don’t have to touch a table that belongs to the other app, which I think is more important.


回答 9

假设您要将模型TheModel从app_a移至app_b。

另一种解决方案是手动更改现有迁移。这样做的想法是,每次在app_a的迁移过程中看到更改TheModel的操作时,都会将该操作复制到app_b的初始迁移的末尾。并且每次在app_a的迁移中看到引用’app_a.TheModel’时,都将其更改为’app_b.TheModel’。

我只是对一个现有项目执行此操作,在该项目中我想将某个模型提取到可重用的应用程序中。程序进行得很顺利。我想如果有从app_b到app_a的引用,事情会更加困难。另外,我为模型手动定义了Meta.db_table,这可能有所帮助。

值得注意的是,您最终会更改迁移历史记录。即使您的数据库已应用原始迁移,也没关系。如果原始迁移和重写迁移最终都具有相同的数据库模式,则这种重写应该可以。

Lets say you are moving model TheModel from app_a to app_b.

An alternate solution is to alter the existing migrations by hand. The idea is that each time you see an operation altering TheModel in app_a’s migrations, you copy that operation to the end of app_b’s initial migration. And each time you see a reference ‘app_a.TheModel’ in app_a’s migrations, you change it to ‘app_b.TheModel’.

I just did this for an existing project, where I wanted to extract a certain model to an reusable app. The procedure went smoothly. I guess things would be much harder if there were references from app_b to app_a. Also, I had a manually defined Meta.db_table for my model which might have helped.

Notably you will end up with altered migration history. This doesn’t matter, even if you have a database with the original migrations applied. If both the original and the rewritten migrations end up with the same database schema, then such rewrite should be OK.


回答 10

  1. 将旧模型的名称更改为“ model_name_old”
  2. 移民
  3. 制作名为“ model_name_new”的新模型,并在相关模型上具有相同的关系(例如,用户模型现在具有user.blog_old和user.blog_new)
  4. 移民
  5. 编写自定义迁移,将所有数据迁移到新模型表
  6. 通过在运行迁移之前和之后将备份与新的数据库副本进行比较来测试这些迁移的难处
  7. 当一切都令人满意时,删除旧模型
  8. 移民
  9. 将新模型更改为正确的名称’model_name_new’->’model_name’
  10. 在登台服务器上测试整个迁移过程
  11. 将您的生产站点关闭几分钟,以便在不干扰用户的情况下运行所有​​迁移

对每个需要移动的模型分别执行此操作。我不建议通过更改为整数然后再返回外键来执行其他答案所说的事情,在迁移之后,新的外键可能会有所不同,并且行的ID可能会有所不同,并且我不想冒任何风险切换回外键时ID不匹配的问题。

  1. change the names of old models to ‘model_name_old’
  2. makemigrations
  3. make new models named ‘model_name_new’ with identical relationships on the related models (eg. user model now has user.blog_old and user.blog_new)
  4. makemigrations
  5. write a custom migration that migrates all the data to the new model tables
  6. test the hell out of these migrations by comparing backups with new db copies before and after running the migrations
  7. when all is satisfactory, delete the old models
  8. makemigrations
  9. change the new models to the correct name ‘model_name_new’ -> ‘model_name’
  10. test the whole slew of migrations on a staging server
  11. take your production site down for a few minutes in order to run all migrations without users interfering

Do this individually for each model that needs to be moved. I wouldn’t suggest doing what the other answer says by changing to integers and back to foreign keys There is a chance that new foreign keys will be different and rows may have different IDs after the migrations and I didn’t want to run any risk of mismatching ids when switching back to foreign keys.


使用SQLAlchemy ORM批量插入

问题:使用SQLAlchemy ORM批量插入

有什么方法可以让SQLAlchemy进行批量插入,而不是插入每个对象。即

在做:

INSERT INTO `foo` (`bar`) VALUES (1), (2), (3)

而不是:

INSERT INTO `foo` (`bar`) VALUES (1)
INSERT INTO `foo` (`bar`) VALUES (2)
INSERT INTO `foo` (`bar`) VALUES (3)

我刚刚将一些代码转换为使用sqlalchemy而不是原始sql,尽管现在使用它起来要好得多,但现在似乎要慢一些(最多10倍),我想知道这是否是原因。

也许我可以更有效地使用会话来改善这种情况。目前,我已经添加了一些东西,autoCommit=False并做了一个session.commit()。尽管如果在其他地方更改了数据库,这似乎会使数据过时,例如,即使我执行新查询,我仍然可以返回旧结果?

谢谢你的帮助!

Is there any way to get SQLAlchemy to do a bulk insert rather than inserting each individual object. i.e.,

doing:

INSERT INTO `foo` (`bar`) VALUES (1), (2), (3)

rather than:

INSERT INTO `foo` (`bar`) VALUES (1)
INSERT INTO `foo` (`bar`) VALUES (2)
INSERT INTO `foo` (`bar`) VALUES (3)

I’ve just converted some code to use sqlalchemy rather than raw sql and although it is now much nicer to work with it seems to be slower now (up to a factor of 10), I’m wondering if this is the reason.

May be I could improve the situation using sessions more efficiently. At the moment I have autoCommit=False and do a session.commit() after I’ve added some stuff. Although this seems to cause the data to go stale if the DB is changed elsewhere, like even if I do a new query I still get old results back?

Thanks for your help!


回答 0

SQLAlchemy在版本中引入了该功能1.0.0

批量操作-SQLAlchemy文档

通过这些操作,您现在可以批量插入或更新!

例如,您可以执行以下操作:

s = Session()
objects = [
    User(name="u1"),
    User(name="u2"),
    User(name="u3")
]
s.bulk_save_objects(objects)
s.commit()

在这里,将制成大量插入物。

SQLAlchemy introduced that in version 1.0.0:

Bulk operations – SQLAlchemy docs

With these operations, you can now do bulk inserts or updates!

For instance, you can do:

s = Session()
objects = [
    User(name="u1"),
    User(name="u2"),
    User(name="u3")
]
s.bulk_save_objects(objects)
s.commit()

Here, a bulk insert will be made.


回答 1

sqlalchemy文档对可用于批量插入的各种技术的性能进行了总结

ORM基本上不是用于高性能批量插入的-这是SQLAlchemy除了将ORM作为一流组件之外还提供Core的全部原因。

对于快速批量插入的用例,ORM所基于的SQL生成和执行系统是Core的一部分。直接使用该系统,我们可以产生与直接使用原始数据库API相比具有竞争力的INSERT。

另外,SQLAlchemy ORM提供了Bulk Operations方法套件,该套件提供了到工作单元过程各部分的挂钩,以便发出基于ORM的自动化程度较低的Core级INSERT和UPDATE构造。

下面的示例说明了基于时间的测试,该测试针对从自动程度最高到最少的几种不同的行插入方法。使用cPython 2.7,可以观察到运行时:

classics-MacBook-Pro:sqlalchemy classic$ python test.py
SQLAlchemy ORM: Total time for 100000 records 12.0471920967 secs
SQLAlchemy ORM pk given: Total time for 100000 records 7.06283402443 secs
SQLAlchemy ORM bulk_save_objects(): Total time for 100000 records 0.856323003769 secs
SQLAlchemy Core: Total time for 100000 records 0.485800027847 secs
sqlite3: Total time for 100000 records 0.487842082977 sec

脚本:

import time
import sqlite3

from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import Column, Integer, String,  create_engine
from sqlalchemy.orm import scoped_session, sessionmaker

Base = declarative_base()
DBSession = scoped_session(sessionmaker())
engine = None


class Customer(Base):
    __tablename__ = "customer"
    id = Column(Integer, primary_key=True)
    name = Column(String(255))


def init_sqlalchemy(dbname='sqlite:///sqlalchemy.db'):
    global engine
    engine = create_engine(dbname, echo=False)
    DBSession.remove()
    DBSession.configure(bind=engine, autoflush=False, expire_on_commit=False)
    Base.metadata.drop_all(engine)
    Base.metadata.create_all(engine)


def test_sqlalchemy_orm(n=100000):
    init_sqlalchemy()
    t0 = time.time()
    for i in xrange(n):
        customer = Customer()
        customer.name = 'NAME ' + str(i)
        DBSession.add(customer)
        if i % 1000 == 0:
            DBSession.flush()
    DBSession.commit()
    print(
        "SQLAlchemy ORM: Total time for " + str(n) +
        " records " + str(time.time() - t0) + " secs")


def test_sqlalchemy_orm_pk_given(n=100000):
    init_sqlalchemy()
    t0 = time.time()
    for i in xrange(n):
        customer = Customer(id=i+1, name="NAME " + str(i))
        DBSession.add(customer)
        if i % 1000 == 0:
            DBSession.flush()
    DBSession.commit()
    print(
        "SQLAlchemy ORM pk given: Total time for " + str(n) +
        " records " + str(time.time() - t0) + " secs")


def test_sqlalchemy_orm_bulk_insert(n=100000):
    init_sqlalchemy()
    t0 = time.time()
    n1 = n
    while n1 > 0:
        n1 = n1 - 10000
        DBSession.bulk_insert_mappings(
            Customer,
            [
                dict(name="NAME " + str(i))
                for i in xrange(min(10000, n1))
            ]
        )
    DBSession.commit()
    print(
        "SQLAlchemy ORM bulk_save_objects(): Total time for " + str(n) +
        " records " + str(time.time() - t0) + " secs")


def test_sqlalchemy_core(n=100000):
    init_sqlalchemy()
    t0 = time.time()
    engine.execute(
        Customer.__table__.insert(),
        [{"name": 'NAME ' + str(i)} for i in xrange(n)]
    )
    print(
        "SQLAlchemy Core: Total time for " + str(n) +
        " records " + str(time.time() - t0) + " secs")


def init_sqlite3(dbname):
    conn = sqlite3.connect(dbname)
    c = conn.cursor()
    c.execute("DROP TABLE IF EXISTS customer")
    c.execute(
        "CREATE TABLE customer (id INTEGER NOT NULL, "
        "name VARCHAR(255), PRIMARY KEY(id))")
    conn.commit()
    return conn


def test_sqlite3(n=100000, dbname='sqlite3.db'):
    conn = init_sqlite3(dbname)
    c = conn.cursor()
    t0 = time.time()
    for i in xrange(n):
        row = ('NAME ' + str(i),)
        c.execute("INSERT INTO customer (name) VALUES (?)", row)
    conn.commit()
    print(
        "sqlite3: Total time for " + str(n) +
        " records " + str(time.time() - t0) + " sec")

if __name__ == '__main__':
    test_sqlalchemy_orm(100000)
    test_sqlalchemy_orm_pk_given(100000)
    test_sqlalchemy_orm_bulk_insert(100000)
    test_sqlalchemy_core(100000)
    test_sqlite3(100000)

The sqlalchemy docs have a writeup on the performance of various techniques that can be used for bulk inserts:

ORMs are basically not intended for high-performance bulk inserts – this is the whole reason SQLAlchemy offers the Core in addition to the ORM as a first-class component.

For the use case of fast bulk inserts, the SQL generation and execution system that the ORM builds on top of is part of the Core. Using this system directly, we can produce an INSERT that is competitive with using the raw database API directly.

Alternatively, the SQLAlchemy ORM offers the Bulk Operations suite of methods, which provide hooks into subsections of the unit of work process in order to emit Core-level INSERT and UPDATE constructs with a small degree of ORM-based automation.

The example below illustrates time-based tests for several different methods of inserting rows, going from the most automated to the least. With cPython 2.7, runtimes observed:

classics-MacBook-Pro:sqlalchemy classic$ python test.py
SQLAlchemy ORM: Total time for 100000 records 12.0471920967 secs
SQLAlchemy ORM pk given: Total time for 100000 records 7.06283402443 secs
SQLAlchemy ORM bulk_save_objects(): Total time for 100000 records 0.856323003769 secs
SQLAlchemy Core: Total time for 100000 records 0.485800027847 secs
sqlite3: Total time for 100000 records 0.487842082977 sec

Script:

import time
import sqlite3

from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import Column, Integer, String,  create_engine
from sqlalchemy.orm import scoped_session, sessionmaker

Base = declarative_base()
DBSession = scoped_session(sessionmaker())
engine = None


class Customer(Base):
    __tablename__ = "customer"
    id = Column(Integer, primary_key=True)
    name = Column(String(255))


def init_sqlalchemy(dbname='sqlite:///sqlalchemy.db'):
    global engine
    engine = create_engine(dbname, echo=False)
    DBSession.remove()
    DBSession.configure(bind=engine, autoflush=False, expire_on_commit=False)
    Base.metadata.drop_all(engine)
    Base.metadata.create_all(engine)


def test_sqlalchemy_orm(n=100000):
    init_sqlalchemy()
    t0 = time.time()
    for i in xrange(n):
        customer = Customer()
        customer.name = 'NAME ' + str(i)
        DBSession.add(customer)
        if i % 1000 == 0:
            DBSession.flush()
    DBSession.commit()
    print(
        "SQLAlchemy ORM: Total time for " + str(n) +
        " records " + str(time.time() - t0) + " secs")


def test_sqlalchemy_orm_pk_given(n=100000):
    init_sqlalchemy()
    t0 = time.time()
    for i in xrange(n):
        customer = Customer(id=i+1, name="NAME " + str(i))
        DBSession.add(customer)
        if i % 1000 == 0:
            DBSession.flush()
    DBSession.commit()
    print(
        "SQLAlchemy ORM pk given: Total time for " + str(n) +
        " records " + str(time.time() - t0) + " secs")


def test_sqlalchemy_orm_bulk_insert(n=100000):
    init_sqlalchemy()
    t0 = time.time()
    n1 = n
    while n1 > 0:
        n1 = n1 - 10000
        DBSession.bulk_insert_mappings(
            Customer,
            [
                dict(name="NAME " + str(i))
                for i in xrange(min(10000, n1))
            ]
        )
    DBSession.commit()
    print(
        "SQLAlchemy ORM bulk_save_objects(): Total time for " + str(n) +
        " records " + str(time.time() - t0) + " secs")


def test_sqlalchemy_core(n=100000):
    init_sqlalchemy()
    t0 = time.time()
    engine.execute(
        Customer.__table__.insert(),
        [{"name": 'NAME ' + str(i)} for i in xrange(n)]
    )
    print(
        "SQLAlchemy Core: Total time for " + str(n) +
        " records " + str(time.time() - t0) + " secs")


def init_sqlite3(dbname):
    conn = sqlite3.connect(dbname)
    c = conn.cursor()
    c.execute("DROP TABLE IF EXISTS customer")
    c.execute(
        "CREATE TABLE customer (id INTEGER NOT NULL, "
        "name VARCHAR(255), PRIMARY KEY(id))")
    conn.commit()
    return conn


def test_sqlite3(n=100000, dbname='sqlite3.db'):
    conn = init_sqlite3(dbname)
    c = conn.cursor()
    t0 = time.time()
    for i in xrange(n):
        row = ('NAME ' + str(i),)
        c.execute("INSERT INTO customer (name) VALUES (?)", row)
    conn.commit()
    print(
        "sqlite3: Total time for " + str(n) +
        " records " + str(time.time() - t0) + " sec")

if __name__ == '__main__':
    test_sqlalchemy_orm(100000)
    test_sqlalchemy_orm_pk_given(100000)
    test_sqlalchemy_orm_bulk_insert(100000)
    test_sqlalchemy_core(100000)
    test_sqlite3(100000)

回答 2

据我所知,没有办法让ORM发出批量插入。我认为根本原因是SQLAlchemy需要跟踪每个对象的身份(即新的主键),而大容量插入会对此产生干扰。例如,假设您的foo表包含一id列并映射到一个Foo类:

x = Foo(bar=1)
print x.id
# None
session.add(x)
session.flush()
# BEGIN
# INSERT INTO foo (bar) VALUES(1)
# COMMIT
print x.id
# 1

由于SQLAlchemy在x.id不发出另一个查询的情况下获取了该值,因此我们可以推断出它直接从该INSERT语句中获取了该值。如果不需要随后通过相同实例访问创建的对象,则可以跳过ORM层进行插入:

Foo.__table__.insert().execute([{'bar': 1}, {'bar': 2}, {'bar': 3}])
# INSERT INTO foo (bar) VALUES ((1,), (2,), (3,))

SQLAlchemy无法将这些新行与任何现有对象匹配,因此您必须重新查询它们以进行任何后续操作。

至于过时的数据,记住该会话没有内置的方式来了解何时在会话外更改数据库是很有帮助的。为了通过现有实例访问外部修改的数据,必须将这些实例标记为expired。默认情况下会发生这种情况session.commit(),但可以通过调用session.expire_all()或手动完成session.expire(instance)。一个例子(省略SQL):

x = Foo(bar=1)
session.add(x)
session.commit()
print x.bar
# 1
foo.update().execute(bar=42)
print x.bar
# 1
session.expire(x)
print x.bar
# 42

session.commit()expires x,因此第一个打印语句隐式打开一个新事务并重新查询x属性。如果注释掉第一个打印语句,您会注意到第二个打印语句现在会选择正确的值,因为直到更新后才会发出新查询。

从事务隔离的角度来看,这是有道理的-您只应在事务之间进行外部修改。如果这给您带来麻烦,建议您弄清或重新考虑应用程序的事务边界,而不要立即进行操作session.expire_all()

As far as I know, there is no way to get the ORM to issue bulk inserts. I believe the underlying reason is that SQLAlchemy needs to keep track of each object’s identity (i.e., new primary keys), and bulk inserts interfere with that. For example, assuming your foo table contains an id column and is mapped to a Foo class:

x = Foo(bar=1)
print x.id
# None
session.add(x)
session.flush()
# BEGIN
# INSERT INTO foo (bar) VALUES(1)
# COMMIT
print x.id
# 1

Since SQLAlchemy picked up the value for x.id without issuing another query, we can infer that it got the value directly from the INSERT statement. If you don’t need subsequent access to the created objects via the same instances, you can skip the ORM layer for your insert:

Foo.__table__.insert().execute([{'bar': 1}, {'bar': 2}, {'bar': 3}])
# INSERT INTO foo (bar) VALUES ((1,), (2,), (3,))

SQLAlchemy can’t match these new rows with any existing objects, so you’ll have to query them anew for any subsequent operations.

As far as stale data is concerned, it’s helpful to remember that the session has no built-in way to know when the database is changed outside of the session. In order to access externally modified data through existing instances, the instances must be marked as expired. This happens by default on session.commit(), but can be done manually by calling session.expire_all() or session.expire(instance). An example (SQL omitted):

x = Foo(bar=1)
session.add(x)
session.commit()
print x.bar
# 1
foo.update().execute(bar=42)
print x.bar
# 1
session.expire(x)
print x.bar
# 42

session.commit() expires x, so the first print statement implicitly opens a new transaction and re-queries x‘s attributes. If you comment out the first print statement, you’ll notice that the second one now picks up the correct value, because the new query isn’t emitted until after the update.

This makes sense from the point of view of transactional isolation – you should only pick up external modifications between transactions. If this is causing you trouble, I’d suggest clarifying or re-thinking your application’s transaction boundaries instead of immediately reaching for session.expire_all().


回答 3

我通常使用add_all

from app import session
from models import User

objects = [User(name="u1"), User(name="u2"), User(name="u3")]
session.add_all(objects)
session.commit()

I usually do it using add_all.

from app import session
from models import User

objects = [User(name="u1"), User(name="u2"), User(name="u3")]
session.add_all(objects)
session.commit()

回答 4

从0.8版开始,直接支持已添加到SQLAlchemy

根据docsconnection.execute(table.insert().values(data))应该可以解决问题。(请注意,这是一样的connection.execute(table.insert(), data)通过将呼叫这导致许多个别行插入executemany)。除了本地连接之外,其他任何方面的性能差异都可能很大。

Direct support was added to SQLAlchemy as of version 0.8

As per the docs, connection.execute(table.insert().values(data)) should do the trick. (Note that this is not the same as connection.execute(table.insert(), data) which results in many individual row inserts via a call to executemany). On anything but a local connection the difference in performance can be enormous.


回答 5

SQLAlchemy在版本中引入了该功能1.0.0

批量操作-SQLAlchemy文档

通过这些操作,您现在可以批量插入或更新!

例如(如果您希望简单表INSERT的开销最小),可以使用Session.bulk_insert_mappings()

loadme = [(1, 'a'),
          (2, 'b'),
          (3, 'c')]
dicts = [dict(bar=t[0], fly=t[1]) for t in loadme]

s = Session()
s.bulk_insert_mappings(Foo, dicts)
s.commit()

或者,如果需要,可以跳过loadme元组,直接将字典写进去dicts(但是我发现将所有的单词遗漏在数据之外并循环加载字典列表会更容易)。

SQLAlchemy introduced that in version 1.0.0:

Bulk operations – SQLAlchemy docs

With these operations, you can now do bulk inserts or updates!

For instance (if you want the lowest overhead for simple table INSERTs), you can use Session.bulk_insert_mappings():

loadme = [(1, 'a'),
          (2, 'b'),
          (3, 'c')]
dicts = [dict(bar=t[0], fly=t[1]) for t in loadme]

s = Session()
s.bulk_insert_mappings(Foo, dicts)
s.commit()

Or, if you want, skip the loadme tuples and write the dictionaries directly into dicts (but I find it easier to leave all the wordiness out of the data and load up a list of dictionaries in a loop).


回答 6

Piere的回答是正确的,但是一个问题是bulk_save_objects,如果您担心的话,默认情况下不会返回对象的主键。设置return_defaultsTrue可得到此行为。

文档在这里

foos = [Foo(bar='a',), Foo(bar='b'), Foo(bar='c')]
session.bulk_save_objects(foos, return_defaults=True)
for foo in foos:
    assert foo.id is not None
session.commit()

Piere’s answer is correct but one issue is that bulk_save_objects by default does not return the primary keys of the objects, if that is of concern to you. Set return_defaults to True to get this behavior.

The documentation is here.

foos = [Foo(bar='a',), Foo(bar='b'), Foo(bar='c')]
session.bulk_save_objects(foos, return_defaults=True)
for foo in foos:
    assert foo.id is not None
session.commit()

回答 7

条条大路通罗马,但其中一些横穿山脉,需要渡轮,但是如果您想快速到达那儿,只需上高速公路。


在这种情况下,高速公路将使用psycopg2execute_batch()功能。该文档说的最好:

当前的实现executemany()(使用非常慈善的轻描淡写)不是特别有效。这些功能可用于加快针对一组参数的语句的重复执行。通过减少服务器往返次数,性能可以比使用服务器好几个数量级。executemany()

在我自己的测试execute_batch()快2倍左右executemany(),并给出配置进行进一步的调整所以page_size的选项(如果你想挤进业绩的最后2-3%的驾驶者)。

如果使用SQLAlchemy,则可以通过use_batch_mode=True在实例化引擎时将其设置为参数来轻松启用相同功能。create_engine()

All Roads Lead to Rome, but some of them crosses mountains, requires ferries but if you want to get there quickly just take the motorway.


In this case the motorway is to use the execute_batch() feature of psycopg2. The documentation says it the best:

The current implementation of executemany() is (using an extremely charitable understatement) not particularly performing. These functions can be used to speed up the repeated execution of a statement against a set of parameters. By reducing the number of server roundtrips the performance can be orders of magnitude better than using executemany().

In my own test execute_batch() is approximately twice as fast as executemany(), and gives the option to configure the page_size for further tweaking (if you want to squeeze the last 2-3% of performance out of the driver).

The same feature can easily be enabled if you are using SQLAlchemy by setting use_batch_mode=True as a parameter when you instantiate the engine with create_engine()


回答 8

这是一种方法:

values = [1, 2, 3]
Foo.__table__.insert().execute([{'bar': x} for x in values])

这样插入:

INSERT INTO `foo` (`bar`) VALUES (1), (2), (3)

参考:SQLAlchemy FAQ包含各种提交方法的基准。

This is a way:

values = [1, 2, 3]
Foo.__table__.insert().execute([{'bar': x} for x in values])

This will insert like this:

INSERT INTO `foo` (`bar`) VALUES (1), (2), (3)

Reference: The SQLAlchemy FAQ includes benchmarks for various commit methods.


回答 9

到目前为止,我发现的最佳答案是在sqlalchemy文档中:

http://docs.sqlalchemy.org/en/latest/faq/performance.html#im-inserting-400-000-rows-with-the-orm-and-it-s-really-slow

有一个完整示例说明了可能的解决方案基准。

如文档所示:

bulk_save_objects不是最佳解决方案,但其性能是正确的。

就可读性而言,第二好的实现是我认为使用SQLAlchemy Core:

def test_sqlalchemy_core(n=100000):
    init_sqlalchemy()
    t0 = time.time()
    engine.execute(
        Customer.__table__.insert(),
            [{"name": 'NAME ' + str(i)} for i in xrange(n)]
    )

文档文章中提供了此功能的上下文。

The best answer I found so far was in sqlalchemy documentation:

http://docs.sqlalchemy.org/en/latest/faq/performance.html#i-m-inserting-400-000-rows-with-the-orm-and-it-s-really-slow

There is a complete example of a benchmark of possible solutions.

As shown in the documentation:

bulk_save_objects is not the best solution but it performance are correct.

The second best implementation in terms of readability I think was with the SQLAlchemy Core:

def test_sqlalchemy_core(n=100000):
    init_sqlalchemy()
    t0 = time.time()
    engine.execute(
        Customer.__table__.insert(),
            [{"name": 'NAME ' + str(i)} for i in xrange(n)]
    )

The context of this function is given in the documentation article.


无法通过套接字’/tmp/mysql.sock连接到本地MySQL服务器

问题:无法通过套接字’/tmp/mysql.sock连接到本地MySQL服务器

当我在测试套件中尝试连接到本地MySQL服务器时,它失败并显示以下错误:

OperationalError: (2002, "Can't connect to local MySQL server through socket '/tmp/mysql.sock' (2)")

但是,我始终可以通过运行命令行mysql程序连接到MySQL 。A ps aux | grep mysql显示服务器正在运行,并 stat /tmp/mysql.sock确认套接字存在。此外,如果我在except该异常的子句中打开调试器,则可以使用完全相同的参数可靠地进行连接。

这个问题可以相当可靠地重现,但是似乎不是100%,因为每当我遇到一个蓝色月亮时,我的测试套件实际上都运行了而没有遇到此错误。当我尝试使用sudo dtruss它时,它没有复制。

所有的客户端代码都在Python中,尽管我不知道这是如何相关的。

切换为使用主机127.0.0.1会产生错误:

DatabaseError: Can't connect to MySQL server on '127.0.0.1' (61)

When I attempted to connect to a local MySQL server during my test suite, it fails with the error:

OperationalError: (2002, "Can't connect to local MySQL server through socket '/tmp/mysql.sock' (2)")

However, I’m able to at all times, connect to MySQL by running the command line mysql program. A ps aux | grep mysql shows the server is running, and stat /tmp/mysql.sock confirm that the socket exists. Further, if I open a debugger in except clause of that exception, I’m able to reliably connect with the exact same parameters.

This issue reproduces fairly reliably, however it doesn’t appear to be 100%, because every once in a blue moon, my test suite does in fact run without hitting this error. When I attempted to run with sudo dtruss it did not reproduce.

All the client code is in Python, though I can’t figure how that’d be relevant.

Switching to use host 127.0.0.1 produces the error:

DatabaseError: Can't connect to MySQL server on '127.0.0.1' (61)

回答 0

sudo /usr/local/mysql/support-files/mysql.server start 

这对我有用。但是,如果这不起作用,请确保mysqld正在运行并尝试连接。

sudo /usr/local/mysql/support-files/mysql.server start 

This worked for me. However, if this doesnt work then make sure that mysqld is running and try connecting.


回答 1

MySQL手册的相关部分在这里。我首先要完成列出的调试步骤。

另外,请记住,在这种情况下,localhost和127.0.0.1是不同的:

  • 如果host设置为localhost,则使用套接字或管道。
  • 如果将host设置为127.0.0.1,则客户端将被强制使用TCP / IP。

因此,例如,您可以检查数据库是否正在侦听TCP连接vi netstat -nlp。它似乎正在侦听TCP连接,因为您说这mysql -h 127.0.0.1很好。要检查是否可以通过套接字连接到数据库,请使用mysql -h localhost

如果以上方法均无济于事,那么您可能需要发布有关MySQL配置,实例化连接的确切方式等的更多详细信息。

The relevant section of the MySQL manual is here. I’d start by going through the debugging steps listed there.

Also, remember that localhost and 127.0.0.1 are not the same thing in this context:

  • If host is set to localhost, then a socket or pipe is used.
  • If host is set to 127.0.0.1, then the client is forced to use TCP/IP.

So, for example, you can check if your database is listening for TCP connections vi netstat -nlp. It seems likely that it IS listening for TCP connections because you say that mysql -h 127.0.0.1 works just fine. To check if you can connect to your database via sockets, use mysql -h localhost.

If none of this helps, then you probably need to post more details about your MySQL config, exactly how you’re instantiating the connection, etc.


回答 2

对我来说,问题是我没有运行mysql服务器。首先运行服务器,然后执行mysql

$ mysql.server start
$ mysql -h localhost -u root -p

For me the problem was I wasn’t running MySQL Server. Run server first and then execute mysql.

$ mysql.server start
$ mysql -h localhost -u root -p

回答 3

当我的开发人员安装了堆栈管理器(如MAMP)并在非标准位置预先安装了MySQL时,就已经在我的商店中看到这种情况。

在您的终端运行

mysql_config --socket

这将为您提供袜子文件的路径。走那条路,并在您的数据库主机参数中使用它。

您需要做的就是指出您的

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.mysql',
        'NAME': 'test',
        'USER': 'test',
        'PASSWORD': 'test',
        'HOST': '/Applications/MAMP/tmp/mysql/mysql.sock',
        'PORT': '',
    },
}

注意

which mysql_config如果您以某种方式在计算机上安装了多个mysql服务器实例,则也可以运行,您可能连接了错误的实例。

I’ve seen this happen at my shop when my devs have a stack manager like MAMP installed that comes preconfigured with MySQL installed in a non standard place.

at your terminal run

mysql_config --socket

that will give you your path to the sock file. take that path and use it in your DATABASES HOST paramater.

What you need to do is point your

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.mysql',
        'NAME': 'test',
        'USER': 'test',
        'PASSWORD': 'test',
        'HOST': '/Applications/MAMP/tmp/mysql/mysql.sock',
        'PORT': '',
    },
}

NOTE

also run which mysql_config if you somehow have multiple instances of mysql server installed on the machine you may be connecting to the wrong one.


回答 4

我只是将HOSTfrom 更改为localhost127.0.0.1并且效果很好:

# settings.py of Django project
...

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.mysql',
        'NAME': 'db_name',
        'USER': 'username',
        'PASSWORD': 'password',
        'HOST': '127.0.0.1',
        'PORT': '',
},
...

I just changed the HOST from localhost to 127.0.0.1 and it works fine:

# settings.py of Django project
...

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.mysql',
        'NAME': 'db_name',
        'USER': 'username',
        'PASSWORD': 'password',
        'HOST': '127.0.0.1',
        'PORT': '',
},
...

回答 5

何时,如果您在Mac OSx中丢失了守护进程mysql,但在其他路径中(例如在private / var中存在),请执行以下命令

1)

ln -s /private/var/mysql/mysql.sock /tmp/mysql.sock

2)使用以下命令重新启动与mysql的连接:

mysql -u username -p -h host databasename

也适用于mariadb

When, if you lose your daemon mysql in mac OSx but is present in other path for exemple in private/var do the following command

1)

ln -s /private/var/mysql/mysql.sock /tmp/mysql.sock

2) restart your connexion to mysql with :

mysql -u username -p -h host databasename

works also for mariadb


回答 6

在终端中运行以下cmd

/ usr / local / mysql / bin / mysqld_safe

然后重新启动机器以生效。有用!!

Run the below cmd in terminal

/usr/local/mysql/bin/mysqld_safe

Then restart the machine to take effect. It works!!


回答 7

使用lsof命令检查mysql进程的打开文件数。

增加打开文件的限制,然后再次运行。

Check number of open files for the mysql process using lsof command.

Increase the open files limit and run again.


回答 8

在尝试了其中一些解决方案但没有成功之后,这对我有用:

  1. 重启系统
  2. mysql.server启动
  3. 成功!

After attempting a few of these solutions and not having any success, this is what worked for me:

  1. Restart system
  2. mysql.server start
  3. Success!

回答 9

这可能是以下问题之一。

  1. 错误的mysql锁定。解决方案:您必须通过以下方式找出正确的mysql套接字,

mysqladmin -p变量| grep套接字

然后将其放在您的数据库连接代码中:

pymysql.connect(db='db', user='user', passwd='pwd', unix_socket="/tmp/mysql.sock")

/tmp/mysql.sock是grep返回的

2.不正确的mysql端口解决方案:您必须找出正确的mysql端口:

mysqladmin -p variables | grep port

然后在您的代码中:

pymysql.connect(db='db', user='user', passwd='pwd', host='localhost', port=3306)

3306是从grep返回的端口

我认为第一种选择可以解决您的问题。

This may be one of following problems.

  1. Incorrect mysql lock. solution: You have to find out the correct mysql socket by,

mysqladmin -p variables | grep socket

and then put it in your db connection code:

pymysql.connect(db='db', user='user', passwd='pwd', unix_socket="/tmp/mysql.sock")

/tmp/mysql.sock is the returned from grep

2.Incorrect mysql port solution: You have to find out the correct mysql port:

mysqladmin -p variables | grep port

and then in your code:

pymysql.connect(db='db', user='user', passwd='pwd', host='localhost', port=3306)

3306 is the port returned from the grep

I think first option will resolve your problem.


回答 10

对于通过自制软件从5.7升级到8.0的用户,此错误很可能是由于升级未完成引起的。就我而言,mysql.server start出现以下错误:

错误!服务器退出而不更新PID文件

然后,我通过检查了日志文件cat /usr/local/var/mysql/YOURS.err | tail -n 50

InnoDB:不支持崩溃后升级。

如果您在同一条船上,请首先mysql@5.7通过自制程序安装,停止服务器,然后再次启动8.0系统。

brew install mysql@5.7

/usr/local/opt/mysql@5.7/bin/mysql.server start
/usr/local/opt/mysql@5.7/bin/mysql.server stop

然后,

mysql.server start

这将使您的MySQL(8.0)再次正常工作。

To those who upgraded from 5.7 to 8.0 via homebrew, this error is likely caused by the upgrade not being complete. In my case, mysql.server start got me the following error:

ERROR! The server quit without updating PID file

I then checked the log file via cat /usr/local/var/mysql/YOURS.err | tail -n 50, and found the following:

InnoDB: Upgrade after a crash is not supported.

If you are on the same boat, first install mysql@5.7 via homebrew, stop the server, and then start the 8.0 system again.

brew install mysql@5.7

/usr/local/opt/mysql@5.7/bin/mysql.server start
/usr/local/opt/mysql@5.7/bin/mysql.server stop

Then,

mysql.server start

This would get your MySQL (8.0) working again.


回答 11

我想我前一段时间也看到了相同的行为,但记不清细节了。
在我们的案例中,问题在于测试运行程序相对于所需的第一次数据库交互(例如,通过在settings.py或某个__init__.py中导入模块)初始化数据库连接的时刻。我将尝试挖掘更多信息,但这可能已经为您解决了。

I think i saw this same behavior some time ago, but can’t remember the details.
In our case, the problem was the moment the testrunner initialises database connections relative to first database interaction required, for instance, by import of a module in settings.py or some __init__.py. I’ll try to digg up some more info, but this might already ring a bell for your case.


回答 12

确保您的/ etc / hosts包含127.0.0.1 localhost其中,并且应该工作正常

Make sure your /etc/hosts has 127.0.0.1 localhost in it and it should work fine


回答 13

我对此有两个偷偷摸摸的猜想

主题1

调查无法访问/tmp/mysql.sock文件的可能性。设置MySQL数据库时,通常将套接字文件站点放入/var/lib/mysql。如果您以身份登录到mysql root@localhost,则您的OS会话需要访问该/tmp文件夹。确保/tmp在操作系统中具有正确的访问权限。另外,请确保sudo用户始终可以读取文件/tmp

主题2

通过访问MySQL 127.0.0.1如果您不注意,则可能会引起一些混乱。怎么样?

在命令行中,如果使用来连接到MySQL 127.0.0.1,则可能需要指定TCP / IP协议。

mysql -uroot -p -h127.0.0.1 --protocol=tcp

或尝试使用DNS名称

mysql -uroot -p -hDNSNAME

这将绕过以身份登录root@localhost,但请确保已root@'127.0.0.1'定义。

下次连接到MySQL时,运行以下命令:

SELECT USER(),CURRENT_USER();

这给你什么?

如果这些函数返回的值相同,则说明您正在按预期进行连接和身份验证。如果值不同,则可能需要创建相应的user root@127.0.0.1

I have two sneaky conjectures on this one

CONJECTURE #1

Look into the possibility of not being able to access the /tmp/mysql.sock file. When I setup MySQL databases, I normally let the socket file site in /var/lib/mysql. If you login to mysql as root@localhost, your OS session needs access to the /tmp folder. Make sure /tmp has the correct access rights in the OS. Also, make sure the sudo user can always read file in /tmp.

CONJECTURE #2

Accessing mysql via 127.0.0.1 can cause some confusion if you are not paying attention. How?

From the command line, if you connect to MySQL with 127.0.0.1, you may need to specify the TCP/IP protocol.

mysql -uroot -p -h127.0.0.1 --protocol=tcp

or try the DNS name

mysql -uroot -p -hDNSNAME

This will bypass logging in as root@localhost, but make sure you have root@'127.0.0.1' defined.

Next time you connect to MySQL, run this:

SELECT USER(),CURRENT_USER();

What does this give you?

  • USER() reports how you attempted to authenticate in MySQL
  • CURRENT_USER() reports how you were allowed to authenticate in MySQL

If these functions return with the same values, then you are connecting and authenticating as expected. If the values are different, you may need to create the corresponding user root@127.0.0.1.


回答 14

遇到了同样的问题。原来mysqld已经停止运行了(我在Mac OSX上)。我重新启动它,错误消失了。

我发现mysqld该链接在很大程度上没有运行: http //dev.mysql.com/doc/refman/5.6/en/can-not-connect-to-server.html

注意第一个技巧!

Had this same problem. Turned out mysqld had stopped running (I’m on Mac OSX). I restarted it and the error went away.

I figured out that mysqld was not running largely because of this link: http://dev.mysql.com/doc/refman/5.6/en/can-not-connect-to-server.html

Notice the first tip!


回答 15

如果出现如下错误:

django.db.utils.OperationalError: (2002, "Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)")

然后只要找到您的mysqld.sock文件位置并将其添加到“主机”即可。

就像我在Linux上使用xampp一样,所以我的mysqld.sock文件在另一个位置。因此它不适用于“ /var/run/mysqld/mysqld.sock

DATABASES = {

    'default': {
        'ENGINE': 'django.db.backends.mysql',
        'NAME': 'asd',
        'USER' : 'root',
        'PASSWORD' : '',
        'HOST' : '/opt/lampp/var/mysql/mysql.sock',
        'PORT' : ''
    }
}

if you get an error like below :

django.db.utils.OperationalError: (2002, "Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)")

Then just find your mysqld.sock file location and add it to “HOST”.

Like i am using xampp on linux so my mysqld.sock file is in another location. so it is not working for ‘/var/run/mysqld/mysqld.sock

DATABASES = {

    'default': {
        'ENGINE': 'django.db.backends.mysql',
        'NAME': 'asd',
        'USER' : 'root',
        'PASSWORD' : '',
        'HOST' : '/opt/lampp/var/mysql/mysql.sock',
        'PORT' : ''
    }
}

回答 16

如果my.cnf中的设置不正确,请检查您的mysql是否未达到最大连接数,或是否未处于某种引导循环中。

使用ps aux | grep mysql检查PID是否正在更改。

Check that your mysql has not reached maximum connections, or is not in some sort of booting loop as happens quite often if the settings are incorrect in my.cnf.

Use ps aux | grep mysql to check if the PID is changing.


回答 17

在网上环顾四周时间太长,无济于事。尝试从命令行键入mysql提示符后,我继续收到此消息:

错误2002(HY000):无法通过套接字’/tmp/mysql.sock’连接到本地MySQL服务器(2)

这是因为我的本地mysql服务器不再运行。为了重新启动服务器,我导航到

shell> cd /user/local/bin

我的mysql.server所在的位置。在这里,只需键入:

shell> mysql.server start

这将重新启动本地mysql服务器。

从那里您可以根据需要重置root密码。

mysql> UPDATE mysql.user SET Password=PASSWORD('MyNewPass')
->                   WHERE User='root';
mysql> FLUSH PRIVILEGES;

Looked around online too long not to contribute. After trying to type in the mysql prompt from the command line, I was continuing to receive this message:

ERROR 2002 (HY000): Can’t connect to local MySQL server through socket ‘/tmp/mysql.sock’ (2)

This was due to the fact that my local mysql server was no longer running. In order to restart the server, I navigated to

shell> cd /user/local/bin

where my mysql.server was located. From here, simply type:

shell> mysql.server start

This will relaunch the local mysql server.

From there you can reset the root password if need be..

mysql> UPDATE mysql.user SET Password=PASSWORD('MyNewPass')
->                   WHERE User='root';
mysql> FLUSH PRIVILEGES;

回答 18

我必须首先找到所有进程ID,以杀死mysql的所有实例:

ps aux | grep MySQL的

然后杀死他们:

杀死-9 {pid}

然后:

mysql.server启动

为我工作。

I had to kill off all instances of mysql by first finding all the process IDs:

ps aux | grep mysql

And then killing them off:

kill -9 {pid}

Then:

mysql.server start

Worked for me.


回答 19

套接字位于/ tmp中。在Unix系统上,由于/ tmp上的模式和所有权,这可能会引起一些问题。但是,只要您告诉我们可以正常使用mysql连接,我想这对您的系统来说不是问题。首要检查应该是将mysql.sock放置在一个更中性的目录中。

问题“随机”发生(或并非每次都发生),这一事实让我认为这可能是服务器问题。

  • 您的/ tmp是位于标准磁盘上,还是位于奇异的挂载上(如RAM中)?

  • / tmp是空的吗?

  • iotop遇到问题时,是否表明您有问题?

The socket is located in /tmp. On Unix system, due to modes & ownerships on /tmp, this could cause some problem. But, as long as you tell us that you CAN use your mysql connexion normally, I guess it is not a problem on your system. A primal check should be to relocate mysql.sock in a more neutral directory.

The fact that the problem occurs “randomly” (or not every time) let me think that it could be a server problem.

  • Is your /tmp located on a standard disk, or on an exotic mount (like in the RAM) ?

  • Is your /tmp empty ?

  • Does iotopshow you something wrong when you encounter the problem ?


回答 20

在“管理数据库连接”对话框中配置数据库连接。选择“标准(TCP / IP)”作为连接方法。

有关更多详细信息,请参见此页面 http://dev.mysql.com/doc/workbench/en/wb-manage-db-connections.html

根据另一页,即使您指定localhost,也会使用套接字文件。

如果您未指定主机名或指定特殊主机名localhost,则使用Unix套接字文件。

它还显示了如何通过运行以下命令来检查服务器:

如果mysqld进程正在运行,则可以通过尝试以下命令进行检查。您的设置中的端口号或Unix套接字文件名可能不同。host_ip代表运行服务器的计算机的IP地址。

shell> mysqladmin version 
shell> mysqladmin variables 
shell> mysqladmin -h `hostname` version variables 
shell> mysqladmin -h `hostname` --port=3306 version 
shell> mysqladmin -h host_ip version 
shell> mysqladmin --protocol=SOCKET --socket=/tmp/mysql.sock version

Configure your DB connection in the ‘Manage DB Connections dialog. Select ‘Standard (TCP/IP)’ as connection method.

See this page for more details http://dev.mysql.com/doc/workbench/en/wb-manage-db-connections.html

According to this other page a socket file is used even if you specify localhost.

A Unix socket file is used if you do not specify a host name or if you specify the special host name localhost.

It also shows how to check on your server by running these commands:

If a mysqld process is running, you can check it by trying the following commands. The port number or Unix socket file name might be different in your setup. host_ip represents the IP address of the machine where the server is running.

shell> mysqladmin version 
shell> mysqladmin variables 
shell> mysqladmin -h `hostname` version variables 
shell> mysqladmin -h `hostname` --port=3306 version 
shell> mysqladmin -h host_ip version 
shell> mysqladmin --protocol=SOCKET --socket=/tmp/mysql.sock version

回答 21

在ubuntu14.04中,您可以执行此操作以解决此问题。

zack@zack:~/pycodes/python-scraping/chapter5$ **mysqladmin -p variables|grep socket**
Enter password: 
| socket                                            | ***/var/run/mysqld/mysqld.sock***                                                                                            |
zack@zack:~/pycodes/python-scraping/chapter5$***ln -s  /var/run/mysqld/mysqld.sock /tmp/mysql.sock***
zack@zack:~/pycodes/python-scraping/chapter5$ ll /tmp/mysql.sock 
lrwxrwxrwx 1 zack zack 27 11 29 13:08 /tmp/mysql.sock -> /var/run/mysqld/mysqld.sock=

in ubuntu14.04 you can do this to slove this problem.

zack@zack:~/pycodes/python-scraping/chapter5$ **mysqladmin -p variables|grep socket**
Enter password: 
| socket                                            | ***/var/run/mysqld/mysqld.sock***                                                                                            |
zack@zack:~/pycodes/python-scraping/chapter5$***ln -s  /var/run/mysqld/mysqld.sock /tmp/mysql.sock***
zack@zack:~/pycodes/python-scraping/chapter5$ ll /tmp/mysql.sock 
lrwxrwxrwx 1 zack zack 27 11月 29 13:08 /tmp/mysql.sock -> /var/run/mysqld/mysqld.sock=

回答 22

对我来说,我确定mysqld已启动,并且命令行mysql可以正常工作。但是httpd服务器显示了问题(无法通过套接字连接到mysql)。

我使用mysqld_safe&启动了该服务。

最后,我发现当我使用服务mysqld start启动mysqld服务时,出现了问题(selinux权限问题),当我修复了selinux问题,并使用“ service mysqld start”启动mysqld时,httpd连接问题消失了。但是,当我使用mysqld_safe&启动mysqld时,可以使用mysqld。(mysql客户端可以正常工作)。但是与httpd连接时仍然存在问题。

For me, I’m sure mysqld is started, and command line mysql can work properly. But the httpd server show the issue(can’t connect to mysql through socket).

I started the service with mysqld_safe&.

finally, I found when I start the mysqld service with service mysqld start, there are issues(selinux permission issue), and when I fix the selinux issue, and start the mysqld with “service mysqld start”, the httpd connection issue disappear. But when I start the mysqld with mysqld_safe&, mysqld can be worked. (mysql client can work properly). But there are still issue when connect with httpd.


回答 23

如果与套接字相关,请阅读此文件

/etc/mysql/my.cnf

并查看什么是标准插座位置。就像这样的一行:

socket = /var/run/mysqld/mysqld.sock

现在为您的shell创建一个别名,例如:

alias mysql="mysql --socket=/var/run/mysqld/mysqld.sock"

这样,您就不需要root特权。

If it’s socket related read this file

/etc/mysql/my.cnf

and see what is the standard socket location. It’s a line like:

socket = /var/run/mysqld/mysqld.sock

now create an alias for your shell like:

alias mysql="mysql --socket=/var/run/mysqld/mysqld.sock"

This way you don’t need root privileges.


回答 24

只需尝试运行 mysqld

在Mac上,这对我不起作用。如果不起作用/usr/local/var/mysql/<your_name>.err,请尝试查看详细的错误日志。

Simply try to run mysqld.

This was what was not working for me on mac. If it doesn’t work try go to /usr/local/var/mysql/<your_name>.err to see detailed error logs.


回答 25

# shell script ,ignore the first 
$ $(dirname `which mysql`)\/mysql.server start

可能会有帮助。

# shell script ,ignore the first 
$ $(dirname `which mysql`)\/mysql.server start

May be helpful.


回答 26

使用通过Homebrew安装的适用于MySQL 8.0.19的MacOS Mojave 10.14.6

  • sudo find / -name my.cnf
  • 文件位于 /usr/local/etc/my.cnf

工作了一段时间,然后最终返回了错误。卸载MySQL的Homebrew版本并直接从.dmg文件安装此处

从那时起,我们一直很高兴地建立联系。

Using MacOS Mojave 10.14.6 for MySQL 8.0.19 installed via Homebrew

  • Ran sudo find / -name my.cnf
  • File found at /usr/local/etc/my.cnf

Worked for a time then eventually the error returned. Uninstalled the Homebrew version of MySQL and installed the .dmg file directly from here

Happily connecting since then.


回答 27

就我而言,帮助编辑文件/etc/mysql/mysql.conf.d/mysqld.cnf并替换以下行:

socket      = /var/run/mysqld/mysqld.sock

socket      = /tmp/mysql.sock

然后,我重新启动服务器,它工作正常。有趣的是,如果我将线路恢复原状并重新启动,它仍然可以工作。

In my case what helped was to edit the file /etc/mysql/mysql.conf.d/mysqld.cnfand replace the line:

socket      = /var/run/mysqld/mysqld.sock

with

socket      = /tmp/mysql.sock

Then I restarted the server and it worked fine. The funny thing is that if I put back the line as it was before and restarted it still worked..


回答 28

我最近也遇到过类似的问题。经历了许多答案。我按照以下步骤操作。

  1. 更改/etc/my.cnf中的套接字路径(因为我反复出现/tmp/mysql.sock错误)引用以更改套接字路径
  2. 运行mysqld_safe以重新启动服务器,因为这是在出现错误时重新启动的推荐方法。引用mysqld_safe

I had faced similar problem recently. Went through many answers. I got it working by following steps.

  1. change the socket path in /etc/my.cnf (as i was repeatedly getting error with /tmp/mysql.sock ) reference to change the socket path
  2. run mysqld_safe to restart the server as it is the recommended way to restart in case of errors. reference to mysqld_safe

回答 29

对我而言,mysql服务器未运行。因此,我通过启动了mysql服务器

mysql.server start

然后

mysql_secure_installation

保护服务器,现在我可以通过访问MySQL服务器

sudo mysql -uroot -p

For me, the mysql server was not running. So, i started the mysql server through

mysql.server start

then

mysql_secure_installation

to secure the server and now I can visit the MySQL server through

sudo mysql -uroot -p


如何仅在内存中运行Django的测试数据库?

问题:如何仅在内存中运行Django的测试数据库?

我的Django单元测试需要很长时间才能运行,因此我正在寻找加快速度的方法。我正在考虑安装SSD,但我知道它也有缺点。当然,我的代码可以做一些事情,但是我正在寻找结构上的修复方法。由于每次都需要重建/向南迁移数据库,因此即使运行单个测试也很慢。所以这是我的主意…

由于我知道测试数据库总是很小,所以为什么不能仅将系统配置为始终将整个测试数据库保留在RAM中?绝对不要触摸磁盘。如何在Django中配置它?我宁愿继续使用MySQL,因为这是我在生产中使用的方式,但是如果使用SQLite  3或其他方法可以简化这一点,我会采用这种方式。

SQLite或MySQL是否可以选择完全在内存中运行?应该可以配置RAM磁盘,然后配置测试数据库以将其数据存储在其中,但是我不确定如何告诉Django / MySQL为特定数据库使用不同的数据目录,特别是因为它不断被删除并重新创建每次运行。(我在Mac FWIW上。)

My Django unit tests take a long time to run, so I’m looking for ways to speed that up. I’m considering installing an SSD, but I know that has its downsides too. Of course, there are things I could do with my code, but I’m looking for a structural fix. Even running a single test is slow since the database needs to be rebuilt / south migrated every time. So here’s my idea…

Since I know the test database will always be quite small, why can’t I just configure the system to always keep the entire test database in RAM? Never touch the disk at all. How do I configure this in Django? I’d prefer to keep using MySQL since that’s what I use in production, but if SQLite 3 or something else makes this easy, I’d go that way.

Does SQLite or MySQL have an option to run entirely in memory? It should be possible to configure a RAM disk and then configure the test database to store its data there, but I’m not sure how to tell Django / MySQL to use a different data directory for a certain database, especially since it keeps getting erased and recreated each run. (I’m on a Mac FWIW.)


回答 0

如果在运行测试时将数据库引擎设置为sqlite3,则Django将使用内存数据库

settings.py在运行测试时使用如下代码将引擎设置为sqlite:

if 'test' in sys.argv:
    DATABASE_ENGINE = 'sqlite3'

或在Django 1.2中:

if 'test' in sys.argv:
    DATABASES['default'] = {'ENGINE': 'sqlite3'}

最后在Django 1.3和1.4中:

if 'test' in sys.argv:
    DATABASES['default'] = {'ENGINE': 'django.db.backends.sqlite3'}

(到Django 1.3并不一定要有完整的后端路径,但是可以使设置向前兼容。)

您还可以添加以下行,以防南向迁移出现问题:

    SOUTH_TESTS_MIGRATE = False

If you set your database engine to sqlite3 when you run your tests, Django will use a in-memory database.

I’m using code like this in my settings.py to set the engine to sqlite when running my tests:

if 'test' in sys.argv:
    DATABASE_ENGINE = 'sqlite3'

Or in Django 1.2:

if 'test' in sys.argv:
    DATABASES['default'] = {'ENGINE': 'sqlite3'}

And finally in Django 1.3 and 1.4:

if 'test' in sys.argv:
    DATABASES['default'] = {'ENGINE': 'django.db.backends.sqlite3'}

(The full path to the backend isn’t strictly necessary with Django 1.3, but makes the setting forward compatible.)

You can also add the following line, in case you are having problems with South migrations:

    SOUTH_TESTS_MIGRATE = False

回答 1

我通常为测试创建一个单独的设置文件,并在测试命令中使用它,例如

python manage.py test --settings=mysite.test_settings myapp

它有两个好处:

  1. 您不必检查testsys.argv中的任何此类神奇词,test_settings.py只需

    from settings import *
    
    # make tests faster
    SOUTH_TESTS_MIGRATE = False
    DATABASES['default'] = {'ENGINE': 'django.db.backends.sqlite3'}

    或者,您可以根据需要进一步调整它,将测试设置与生产设置完全分开。

  2. 另一个好处是您可以使用生产数据库引擎而不是sqlite3进行测试,从而避免了细微的错误,因此在开发使用时

    python manage.py test --settings=mysite.test_settings myapp

    并在提交代码之前运行一次

    python manage.py test myapp

    只是为了确保所有测试都通过了。

I usually create a separate settings file for tests and use it in test command e.g.

python manage.py test --settings=mysite.test_settings myapp

It has two benefits:

  1. You don’t have to check for test or any such magic word in sys.argv, test_settings.py can simply be

    from settings import *
    
    # make tests faster
    SOUTH_TESTS_MIGRATE = False
    DATABASES['default'] = {'ENGINE': 'django.db.backends.sqlite3'}
    

    Or you can further tweak it for your needs, cleanly separating test settings from production settings.

  2. Another benefit is that you can run test with production database engine instead of sqlite3 avoiding subtle bugs, so while developing use

    python manage.py test --settings=mysite.test_settings myapp
    

    and before committing code run once

    python manage.py test myapp
    

    just to be sure that all test are really passing.


回答 2

MySQL支持称为“ MEMORY”的存储引擎,您可以在数据库config(settings.py)中对其进行配置,如下所示:

    'USER': 'root',                      # Not used with sqlite3.
    'PASSWORD': '',                  # Not used with sqlite3.
    'OPTIONS': {
        "init_command": "SET storage_engine=MEMORY",
    }

请注意,MEMORY存储引擎不支持blob /文本列,因此,如果您使用django.db.models.TextField此功能,则将无法使用。

MySQL supports a storage engine called “MEMORY”, which you can configure in your database config (settings.py) as such:

    'USER': 'root',                      # Not used with sqlite3.
    'PASSWORD': '',                  # Not used with sqlite3.
    'OPTIONS': {
        "init_command": "SET storage_engine=MEMORY",
    }

Note that the MEMORY storage engine doesn’t support blob / text columns, so if you’re using django.db.models.TextField this won’t work for you.


回答 3

我无法回答您的主要问题,但是您可以做一些事情来加快速度。

首先,请确保您的MySQL数据库已设置为使用InnoDB。然后,它可以使用事务在每次测试之前回滚db的状态,以我的经验,这导致了极大的提速。您可以在settings.py中传递数据库init命令(Django 1.2语法):

DATABASES = {
    'default': {
            'ENGINE':'django.db.backends.mysql',
            'HOST':'localhost',
            'NAME':'mydb',
            'USER':'whoever',
            'PASSWORD':'whatever',
            'OPTIONS':{"init_command": "SET storage_engine=INNODB" } 
        }
    }

其次,您不需要每次都运行South迁移。设置SOUTH_TESTS_MIGRATE = False在你的settings.py和数据库将与普通的执行syncdb,这将是比通过所有历史悠久的迁移运行更快创建。

I can’t answer your main question, but there are a couple of things that you can do to speed things up.

Firstly, make sure that your MySQL database is set up to use InnoDB. Then it can use transactions to rollback the state of the db before each test, which in my experience has led to a massive speed-up. You can pass a database init command in your settings.py (Django 1.2 syntax):

DATABASES = {
    'default': {
            'ENGINE':'django.db.backends.mysql',
            'HOST':'localhost',
            'NAME':'mydb',
            'USER':'whoever',
            'PASSWORD':'whatever',
            'OPTIONS':{"init_command": "SET storage_engine=INNODB" } 
        }
    }

Secondly, you don’t need to run the South migrations each time. Set SOUTH_TESTS_MIGRATE = False in your settings.py and the database will be created with plain syncdb, which will be much quicker than running through all the historic migrations.


回答 4

您可以进行两次调整:

  • 使用事务表:在每个TestCase之后,将使用数据库回滚来设置初始固定装置状态。
  • 将您的数据库数据目录放在ramdisk上:就数据库创建而言,您将获得很多收益,并且运行测试会更快。

我正在使用这两种技巧,我很高兴。

如何在Ubuntu上为MySQL设置它:

$ sudo service mysql stop
$ sudo cp -pRL /var/lib/mysql /dev/shm/mysql

$ vim /etc/mysql/my.cnf
# datadir = /dev/shm/mysql
$ sudo service mysql start

当心,这只是为了测试,从内存中重新启动数据库后,丢失了!

You can do double tweaking:

  • use transactional tables: initial fixtures state will be set using database rollback after every TestCase.
  • put your database data dir on ramdisk: you will gain much as far as database creation is concerned and also running test will be faster.

I’m using both tricks and I’m quite happy.

How to set up it for MySQL on Ubuntu:

$ sudo service mysql stop
$ sudo cp -pRL /var/lib/mysql /dev/shm/mysql

$ vim /etc/mysql/my.cnf
# datadir = /dev/shm/mysql
$ sudo service mysql start

Beware, it’s just for testing, after reboot your database from memory is lost!


回答 5

另一种方法:让另一个MySQL实例在使用RAM磁盘的tempf中运行。这篇博客文章中的说明:加速MySQL以在Django中进行测试

优点:

  • 您使用与生产服务器完全相同的数据库
  • 无需更改默认的mysql配置

Another approach: have another instance of MySQL running in a tempfs that uses a RAM Disk. Instructions in this blog post: Speeding up MySQL for testing in Django.

Advantages:

  • You use the exactly same database that your production server uses
  • no need to change your default mysql configuration

回答 6

通过扩展Anurag的答案,我通过创建相同的test_settings并将以下内容添加到manage.py来简化了过程

if len(sys.argv) > 1 and sys.argv[1] == "test":
    os.environ.setdefault("DJANGO_SETTINGS_MODULE", "mysite.test_settings")
else:
    os.environ.setdefault("DJANGO_SETTINGS_MODULE", "mysite.settings")

似乎更干净,因为sys已经导入并且manage.py仅通过命令行使用,因此无需弄乱设置

Extending on Anurag’s answer I simplified the process by creating the same test_settings and adding the following to manage.py

if len(sys.argv) > 1 and sys.argv[1] == "test":
    os.environ.setdefault("DJANGO_SETTINGS_MODULE", "mysite.test_settings")
else:
    os.environ.setdefault("DJANGO_SETTINGS_MODULE", "mysite.settings")

seems cleaner since sys is already imported and manage.py is only used via command line, so no need to clutter up settings


回答 7

在您的下方使用 setting.py

DATABASES['default']['ENGINE'] = 'django.db.backends.sqlite3'

Use below in your setting.py

DATABASES['default']['ENGINE'] = 'django.db.backends.sqlite3'

ImportError:没有名为MySQLdb的模块

问题:ImportError:没有名为MySQLdb的模块

我指的是以下教程来为我的Web应用程序创建登录页面。 http://code.tutsplus.com/tutorials/intro-to-flask-signing-in-and-out–net-29982

我的数据库有问题。我正在

ImportError: No module named MySQLdb

当我执行

http://127.0.0.1:5000/testdb

我已经尝试了所有可能的方法来安装python mysql,这是本教程中提到的一种,easy_install,sudo apt-get install。

我已经在虚拟环境中安装了mysql。我的目录结构与本教程中说明的目录结构相同。该模块已成功安装在我的系统中,但仍然出现此错误。

请帮忙。是什么原因造成的。

I am referring the following tutorial to make a login page for my web application. http://code.tutsplus.com/tutorials/intro-to-flask-signing-in-and-out–net-29982

I am having issue with the database. I am getting an

ImportError: No module named MySQLdb

when I execute

http://127.0.0.1:5000/testdb

I have tried all possible ways to install python mysql, the one mentioned in the tutorial, easy_install, sudo apt-get install.

I have installed mysql in my virtual env. My directory structure is just the same as whats explained in the tutorial. The module is sucessfully installed in my system and still I am getting this error.

Please help. What could be causing this.


回答 0

如果您在编译二进制扩展名时遇到问题,或者在无法扩展的平台上,则可以尝试使用纯python PyMySQL绑定。

只需pip install pymysql切换您的SQLAlchemy URI即可,如下所示:

SQLALCHEMY_DATABASE_URI = 'mysql+pymysql://.....'

您还可以尝试其他一些驱动程序

If you’re having issues compiling the binary extension, or on a platform where you cant, you can try using the pure python PyMySQL bindings.

Simply pip install pymysql and switch your SQLAlchemy URI to start like this:

SQLALCHEMY_DATABASE_URI = 'mysql+pymysql://.....'

There are some other drivers you could also try.


回答 1

或尝试以下方法:

apt-get install python-mysqldb

Or try this:

apt-get install python-mysqldb

回答 2

你可以尝试

pip install mysqlclient

may you try

pip install mysqlclient

回答 3

我的问题是:

return __import__('MySQLdb')
ImportError: No module named MySQLdb

和我的决议:

pip install MySQL-python
yum install mysql-devel.x86_64

在开始的时候,我刚刚安装了MySQL-python,但是问题仍然存在。因此,我认为如果发生此问题,您还应该考虑mysql-devel。希望这可以帮助。

My issue is :

return __import__('MySQLdb')
ImportError: No module named MySQLdb

and my resolution :

pip install MySQL-python
yum install mysql-devel.x86_64

at the very beginning, i just installed MySQL-python, but the issue still existed. So i think if this issue happened, you should also take mysql-devel into consideration. Hope this helps.


回答 4

在研究SQLAlchemy时遇到了这个问题。SQLAlchemy用于MySQL的默认方言是mysql+mysqldb

engine = create_engine('mysql+mysqldb://scott:tiger@localhost/foo')

No module named MySQLdb执行上述命令时出现“ ”错误。要修复它,我安装了mysql-python模块,此问题已解决。

sudo pip install mysql-python

I got this issue when I was working on SQLAlchemy. The default dialect used by SQLAlchemy for MySQL is mysql+mysqldb.

engine = create_engine('mysql+mysqldb://scott:tiger@localhost/foo')

I got the “No module named MySQLdb” error when the above command was executed. To fix it I installed the mysql-python module and the issue was fixed.

sudo pip install mysql-python

回答 5

根据我的经验,它也取决于Python版本。

如果您使用的是Python 3,则@DazWorrall答案对我来说效果很好。

但是,如果您使用的是Python 2,则应该

sudo pip install mysql-python

这将安装“ MySQLdb”模块,而无需更改SQLAlchemy URI。

It depends on Python Version as well in my experience.

If you are using Python 3, @DazWorrall answer worked fine for me.

However, if you are using Python 2, you should

sudo pip install mysql-python

which would install ‘MySQLdb’ module without having to change the SQLAlchemy URI.


回答 6

所以我花了大约5个小时试图弄清楚在尝试运行时如何处理此问题

./manage.py makemigrations

使用Ubuntu Server LTS 16.1,完整的LAMP堆栈,Apache2 MySql 5.7 PHP 7 Python 3和Django 1.10.2,我确实很难找到一个好的答案。实际上,我仍然不满意,但是对我有用的唯一解决方案是……

sudo apt-get install build-essential python-dev libapache2-mod-wsgi-py3 libmysqlclient-dev

其次(从虚拟环境内部)

pip install mysqlclient

我真的不喜欢在尝试设置新的Web服务器时必须使用dev install,但是不幸的是,这种配置是我唯一可以采用的舒适方式。

So I spent about 5 hours trying to figure out how to deal with this issue when trying to run

./manage.py makemigrations

With Ubuntu Server LTS 16.1, a full LAMP stack, Apache2 MySql 5.7 PHP 7 Python 3 and Django 1.10.2 I really struggled to find a good answer to this. In fact, I am still not satisfied, but the ONLY solution that worked for me is this…

sudo apt-get install build-essential python-dev libapache2-mod-wsgi-py3 libmysqlclient-dev

followed by (from inside the virtual environment)

pip install mysqlclient

I really dislike having to use dev installs when I am trying to set up a new web server, but unfortunately this configuration was the only mostly comfortable path I could take.


回答 7

尽管@Edward van Kuik答案是正确的,但并未考虑virtualenv v1.7及更高版本的问题

特别是在Ubuntu 上python-mysqldb通过via apt进行安装时,将其放在下/usr/lib/pythonX.Y/dist-packages,但virtualenv的默认情况下不包含此路径。sys.path

因此,要解决此问题,您应该通过运行类似以下内容的系统包来创建您的virtualenv:

virtualenv --system-site-packages .venv

While @Edward van Kuik‘s answer is correct, it doesn’t take into account an issue with virtualenv v1.7 and above.

In particular installing python-mysqldb via apt on Ubuntu put it under /usr/lib/pythonX.Y/dist-packages, but this path isn’t included by default in the virtualenv’s sys.path.

So to resolve this, you should create your virtualenv with system packages by running something like:

virtualenv --system-site-packages .venv


回答 8

有太多与权限有关的错误,什么也没有。您可能想试试这个:

xcode-select --install

Got so many errors related to permissions and what not. You may wanna try this :

xcode-select --install

回答 9

yum install MySQL-python.x86_64

为我工作。

yum install MySQL-python.x86_64

worked for me.


回答 10

在ubuntu 20中,您可以尝试以下操作:

sudo apt-get install libmysqlclient-dev
sudo apt-get install gcc
pip install mysqlclient

In ubuntu 20 , you can try this :

sudo apt-get install libmysqlclient-dev
sudo apt-get install gcc
pip install mysqlclient

将Python datetime.datetime对象插入MySQL

问题:将Python datetime.datetime对象插入MySQL

我在MySQL表中有一个日期列。我想在datetime.datetime()此列中插入一个对象。我应该在execute语句中使用什么?

我努力了:

now = datetime.datetime(2009,5,5)

cursor.execute("INSERT INTO table
(name, id, datecolumn) VALUES (%s, %s
, %s)",("name", 4,now))

我收到以下错误消息:"TypeError: not all arguments converted during string formatting" 应该用什么代替%s

I have a date column in a MySQL table. I want to insert a datetime.datetime() object into this column. What should I be using in the execute statement?

I have tried:

now = datetime.datetime(2009,5,5)

cursor.execute("INSERT INTO table
(name, id, datecolumn) VALUES (%s, %s
, %s)",("name", 4,now))

I am getting an error as: "TypeError: not all arguments converted during string formatting" What should I use instead of %s?


回答 0

对于时间字段,请使用:

import time    
time.strftime('%Y-%m-%d %H:%M:%S')

我认为strftime也适用于日期时间。

For a time field, use:

import time    
time.strftime('%Y-%m-%d %H:%M:%S')

I think strftime also applies to datetime.


回答 1

您最有可能收到TypeError,因为您需要在datecolumn值前后加上引号。

尝试:

now = datetime.datetime(2009, 5, 5)

cursor.execute("INSERT INTO table (name, id, datecolumn) VALUES (%s, %s, '%s')",
               ("name", 4, now))

关于格式,我成功使用了上面的命令(包括毫秒)和以下命令:

now.strftime('%Y-%m-%d %H:%M:%S')

希望这可以帮助。

You are most likely getting the TypeError because you need quotes around the datecolumn value.

Try:

now = datetime.datetime(2009, 5, 5)

cursor.execute("INSERT INTO table (name, id, datecolumn) VALUES (%s, %s, '%s')",
               ("name", 4, now))

With regards to the format, I had success with the above command (which includes the milliseconds) and with:

now.strftime('%Y-%m-%d %H:%M:%S')

Hope this helps.


回答 2

尝试使用now.date()获取Date对象而不是获取对象DateTime

如果那不起作用,那么将其转换为字符串应该起作用:

now = datetime.datetime(2009,5,5)
str_now = now.date().isoformat()
cursor.execute('INSERT INTO table (name, id, datecolumn) VALUES (%s,%s,%s)', ('name',4,str_now))

Try using now.date() to get a Date object rather than a DateTime.

If that doesn’t work, then converting that to a string should work:

now = datetime.datetime(2009,5,5)
str_now = now.date().isoformat()
cursor.execute('INSERT INTO table (name, id, datecolumn) VALUES (%s,%s,%s)', ('name',4,str_now))

回答 3

使用Python方法datetime.strftime(format),其中format = '%Y-%m-%d %H:%M:%S'

import datetime

now = datetime.datetime.utcnow()

cursor.execute("INSERT INTO table (name, id, datecolumn) VALUES (%s, %s, %s)",
               ("name", 4, now.strftime('%Y-%m-%d %H:%M:%S')))

时区

如果需要考虑时区,则可以按如下所示为UTC设置MySQL时区:

cursor.execute("SET time_zone = '+00:00'")

时区可以在Python中设置:

now = datetime.datetime.utcnow().replace(tzinfo=datetime.timezone.utc)

MySQL文档

MySQL可以使用以下格式识别DATETIME和TIMESTAMP值:

作为任一字符串 ‘YYYY-MM-DD HH:MM:SS’‘YY-MM-DD HH:MM:SS’ 格式。这里也允许使用“宽松”语法:任何标点符号都可以用作日期部分或时间部分之间的分隔符。例如,“ 2012-12-31 11:30:45”,“ 2012 ^ 12 ^ 31 11 + 30 + 45”,“ 2012/12/31 11 * 30 * 45”和“ 2012 @ 12 @ 31 11” ^ 30 ^ 45’是等效的。

在日期和时间部分与小数秒部分之间唯一识别的分隔符是小数点。

日期和时间部分可以用T分隔而不是空格。例如,“ 2012-12-31 11:30:45”与“ 2012-12-31T11:30:45”等效。

如果字符串不带分隔符,则格式为“ YYYYMMDDHHMMSS”或“ YYMMDDHHMMSS”,但前提是该字符串应作为日期使用。例如,“ 20070523091528”和“ 070523091528”被解释为“ 2007-05-23 09:15:28”,但“ 071122129015”是非法的(具有无意义的分钟部分),并变为“ 0000-00-00 00”: 00:00’。

如果数字是日期,则以YYYYMMDDHHMMSS或YYMMDDHHMMSS格式表示。例如,将19830905132800和830905132800解释为“ 1983-09-05 13:28:00”。

Use Python method datetime.strftime(format), where format = '%Y-%m-%d %H:%M:%S'.

import datetime

now = datetime.datetime.utcnow()

cursor.execute("INSERT INTO table (name, id, datecolumn) VALUES (%s, %s, %s)",
               ("name", 4, now.strftime('%Y-%m-%d %H:%M:%S')))

Timezones

If timezones are a concern, the MySQL timezone can be set for UTC as follows:

cursor.execute("SET time_zone = '+00:00'")

And the timezone can be set in Python:

now = datetime.datetime.utcnow().replace(tzinfo=datetime.timezone.utc)

MySQL Documentation

MySQL recognizes DATETIME and TIMESTAMP values in these formats:

As a string in either ‘YYYY-MM-DD HH:MM:SS’ or ‘YY-MM-DD HH:MM:SS’ format. A “relaxed” syntax is permitted here, too: Any punctuation character may be used as the delimiter between date parts or time parts. For example, ‘2012-12-31 11:30:45’, ‘2012^12^31 11+30+45’, ‘2012/12/31 11*30*45’, and ‘2012@12@31 11^30^45’ are equivalent.

The only delimiter recognized between a date and time part and a fractional seconds part is the decimal point.

The date and time parts can be separated by T rather than a space. For example, ‘2012-12-31 11:30:45’ ‘2012-12-31T11:30:45’ are equivalent.

As a string with no delimiters in either ‘YYYYMMDDHHMMSS’ or ‘YYMMDDHHMMSS’ format, provided that the string makes sense as a date. For example, ‘20070523091528’ and ‘070523091528’ are interpreted as ‘2007-05-23 09:15:28’, but ‘071122129015’ is illegal (it has a nonsensical minute part) and becomes ‘0000-00-00 00:00:00’.

As a number in either YYYYMMDDHHMMSS or YYMMDDHHMMSS format, provided that the number makes sense as a date. For example, 19830905132800 and 830905132800 are interpreted as ‘1983-09-05 13:28:00’.


回答 4

您要连接到哪个数据库?我知道Oracle对日期格式可能很挑剔,并且喜欢ISO 8601格式。

**注意:糟糕,我刚读过您在MySQL上。只需格式化日期并尝试将其作为单独的直接SQL调用进行测试即可。

在Python中,您可以获得一个ISO日期,例如

now.isoformat()

例如,Oracle喜欢日期,例如

insert into x values(99, '31-may-09');

根据您的数据库,如果是Oracle,则可能需要TO_DATE它:

insert into x
values(99, to_date('2009/05/31:12:00:00AM', 'yyyy/mm/dd:hh:mi:ssam'));

TO_DATE的一般用法是:

TO_DATE(<string>, '<format>')

如果使用另一个数据库(我看到了光标并认为是Oracle;可能是错误的),请检查其日期格式工具。对于MySQL,它是DATE_FORMAT(),对于SQL Server,它是CONVERT。

另外,使用SQLAlchemy之类的工具将消除此类差异,并使您的生活变得轻松。

What database are you connecting to? I know Oracle can be picky about date formats and likes ISO 8601 format.

**Note: Oops, I just read you are on MySQL. Just format the date and try it as a separate direct SQL call to test.

In Python, you can get an ISO date like

now.isoformat()

For instance, Oracle likes dates like

insert into x values(99, '31-may-09');

Depending on your database, if it is Oracle you might need to TO_DATE it:

insert into x
values(99, to_date('2009/05/31:12:00:00AM', 'yyyy/mm/dd:hh:mi:ssam'));

The general usage of TO_DATE is:

TO_DATE(<string>, '<format>')

If using another database (I saw the cursor and thought Oracle; I could be wrong) then check their date format tools. For MySQL it is DATE_FORMAT() and SQL Server it is CONVERT.

Also using a tool like SQLAlchemy will remove differences like these and make your life easy.


回答 5

如果您仅使用python datetime.date(而不是完整的datetime.datetime),则将日期转换为字符串。这非常简单,对我有用(mysql,python 2.7,Ubuntu)。该列published_date是一个MySQL日期字段,python变量publish_datedatetime.date

# make the record for the passed link info
sql_stmt = "INSERT INTO snippet_links (" + \
    "link_headline, link_url, published_date, author, source, coco_id, link_id)" + \
    "VALUES(%s, %s, %s, %s, %s, %s, %s) ;"

sql_data = ( title, link, str(publish_date), \
             author, posted_by, \
             str(coco_id), str(link_id) )

try:
    dbc.execute(sql_stmt, sql_data )
except Exception, e:
    ...

If you’re just using a python datetime.date (not a full datetime.datetime), just cast the date as a string. This is very simple and works for me (mysql, python 2.7, Ubuntu). The column published_date is a MySQL date field, the python variable publish_date is datetime.date.

# make the record for the passed link info
sql_stmt = "INSERT INTO snippet_links (" + \
    "link_headline, link_url, published_date, author, source, coco_id, link_id)" + \
    "VALUES(%s, %s, %s, %s, %s, %s, %s) ;"

sql_data = ( title, link, str(publish_date), \
             author, posted_by, \
             str(coco_id), str(link_id) )

try:
    dbc.execute(sql_stmt, sql_data )
except Exception, e:
    ...

回答 6

当烦恼到T-SQL

这失败了:

select CONVERT(datetime,'2019-09-13 09:04:35.823312',21)

这有效:

select CONVERT(datetime,'2019-09-13 09:04:35.823',21)

简单的方法:

regexp = re.compile(r'\.(\d{6})')
def to_splunk_iso(dt):
    """Converts the datetime object to Splunk isoformat string."""
    # 6-digits string.
    microseconds = regexp.search(dt).group(1)
    return regexp.sub('.%d' % round(float(microseconds) / 1000), dt)

when iserting into t-sql

this fails:

select CONVERT(datetime,'2019-09-13 09:04:35.823312',21)

this works:

select CONVERT(datetime,'2019-09-13 09:04:35.823',21)

easy way:

regexp = re.compile(r'\.(\d{6})')
def to_splunk_iso(dt):
    """Converts the datetime object to Splunk isoformat string."""
    # 6-digits string.
    microseconds = regexp.search(dt).group(1)
    return regexp.sub('.%d' % round(float(microseconds) / 1000), dt)

Mac OS X-EnvironmentError:找不到mysql_config

问题:Mac OS X-EnvironmentError:找不到mysql_config

首先,是的,我已经看到了:

pip安装mysql-python失败,并显示EnvironmentError:找不到mysql_config

问题

我正在尝试在Google App Engine项目上使用Django。但是,由于以下原因,服务器无法正常启动,因此无法启动。

ImproperlyConfigured("Error loading MySQLdb module: %s" % e)
ImproperlyConfigured: Error loading MySQLdb module: No module named MySQLdb

我做了一些研究,所有这些都指出必须安装Mysql-python,因为它显然不在我的系统上。我实际上尝试卸载它,并得到了:

Cannot uninstall requirement mysql-python, not installed

每当我真正尝试通过以下方式安装时:

sudo pip install MySQL-python

我收到一条错误消息:

raise EnvironmentError("%s not found" % (mysql_config.path,))
EnvironmentError: mysql_config not found

我已经尝试运行:

export PATH=$PATH:/usr/local/mysql/bin

但这似乎没有帮助,因为我再次运行了安装命令,但仍然失败。

有任何想法吗?

请注意,我不在virtualenv中。

First off, yeah, I’ve already seen this:

pip install mysql-python fails with EnvironmentError: mysql_config not found

The problem

I am trying to use Django on a Google App Engine project. However, I haven’t been able to get started as the server fails to start properly due to:

ImproperlyConfigured("Error loading MySQLdb module: %s" % e)
ImproperlyConfigured: Error loading MySQLdb module: No module named MySQLdb

I did some research and it all pointed to having to install Mysql-python, as apparently it isn’t on my system. I actually tried uninstalling it and got this:

Cannot uninstall requirement mysql-python, not installed

Whenever I actually do try to install via:

sudo pip install MySQL-python

I get an error stating:

raise EnvironmentError("%s not found" % (mysql_config.path,))
EnvironmentError: mysql_config not found

I’ve already tried running:

export PATH=$PATH:/usr/local/mysql/bin

but that didn’t seem to help, as I ran the installation command again and it still failed.

Any ideas?

Please note I’m not in a virtualenv.


回答 0

好吧,首先,让我检查一下我是否和您在同一页面上:

  • 您安装了python
  • 你做了 brew install mysql
  • 你做了 export PATH=$PATH:/usr/local/mysql/bin
  • 最后,您做到了pip install MySQL-Python(或者pip3 install mysqlclient如果使用python 3)

如果您以相同的顺序执行了所有这些步骤,但是仍然出现错误,请继续阅读,如果您没有按照这些确切的步骤尝试,那么请从头开始。

因此,您按照这些步骤进行操作,仍然遇到错误,可以尝试以下操作:

  1. 尝试which mysql_config从bash 运行。可能找不到。这就是为什么构建也找不到它的原因。尝试运行locate mysql_config,看看是否有任何返回。该二进制文件的路径需要在您外壳的$ PATH环境变量中,或者必须在该模块的setup.py文件中显式地存在(假设它正在某个特定位置查找该文件)。

  2. 除了使用MySQL-Python,还可以尝试使用“ mysql-connector-python”,它可以使用安装pip install mysql-connector-python。有关此的更多信息,请参见此处此处

  3. 手动找到“ mysql / bin”,“ mysql_config”和“ MySQL-Python”的位置,并将所有这些都添加到$ PATH环境变量中。

  4. 如果以上所有步骤均失败,则可以尝试使用MacPorts安装“ mysql”,在这种情况下,文件“ mysql_config”实际上将被称为“ mysql_config5”,在这种情况下,必须在安装后执行以下操作:export PATH=$PATH:/opt/local/lib/mysql5/bin。您可以在此处找到更多详细信息。

注意1:我见过有人说安装python-dev并提供了libmysqlclient-dev帮助,但是我不知道这些软件包是否在Mac OS上可用。

注意2:此外,请确保尝试以root用户身份运行命令。

我得到了我的答案,从(除了我的大脑)这些地方(也许你可以看看他们,看看它是否会有所帮助):1234

我希望我能提供帮助,并且很高兴知道其中任何一个是否有效。祝好运。

Ok, well, first of all, let me check if I am on the same page as you:

  • You installed python
  • You did brew install mysql
  • You did export PATH=$PATH:/usr/local/mysql/bin
  • And finally, you did pip install MySQL-Python (or pip3 install mysqlclient if using python 3)

If you did all those steps in the same order, and you still got an error, read on to the end, if, however, you did not follow these exact steps try, following them from the very beginning.

So, you followed the steps, and you’re still geting an error, well, there are a few things you could try:

  1. Try running which mysql_config from bash. It probably won’t be found. That’s why the build isn’t finding it either. Try running locate mysql_config and see if anything comes back. The path to this binary needs to be either in your shell’s $PATH environment variable, or it needs to be explicitly in the setup.py file for the module assuming it’s looking in some specific place for that file.

  2. Instead of using MySQL-Python, try using ‘mysql-connector-python’, it can be installed using pip install mysql-connector-python. More information on this can be found here and here.

  3. Manually find the location of ‘mysql/bin’, ‘mysql_config’, and ‘MySQL-Python’, and add all these to the $PATH environment variable.

  4. If all above steps fail, then you could try installing ‘mysql’ using MacPorts, in which case the file ‘mysql_config’ would actually be called ‘mysql_config5’, and in this case, you would have to do this after installing: export PATH=$PATH:/opt/local/lib/mysql5/bin. You can find more details here.

Note1: I’ve seen some people saying that installing python-dev and libmysqlclient-dev also helped, however I do not know if these packages are available on Mac OS.

Note2: Also, make sure to try running the commands as root.

I got my answers from (besides my brain) these places (maybe you could have a look at them, to see if it would help): 1, 2, 3, 4.

I hoped I helped, and would be happy to know if any of this worked, or not. Good luck.


回答 1

我一直在调试此问题-3小时17分钟。令我特别烦恼的是,我已经通过先前的uni工作在系统上安装了sql,但是pip / pip3却无法识别它。这些线程以及我在互联网上搜索过的许多其他线程有助于消除问题,但实际上并没有解决问题。

回答

Pip在相对于Macintosh HD @ 的Homebrew目录中寻找mysql二进制文件

/usr/local/Cellar/

所以我发现这需要您进行一些更改

步骤1:如果尚未下载MySql,请下载https://dev.mysql.com/downloads/

步骤2:相对于Macintosh HD和cd 找到它

/usr/local/mysql/bin

第3步:打开终端并使用所选的文本编辑器后-我本人是个Neovim家伙,所以我打了一下(Mac并不会自动附带……另一天的故事)

nvim mysql_config

第4步:您将在第112行看到

# Create options 
libs="-L$pkglibdir"
libs="$libs -l "

改成

# Create options 
libs="-L$pkglibdir"
libs="$libs -lmysqlclient -lssl -lcrypto"

*您会注意到该文件具有只读访问权限,因此,如果您使用vim或neovim

:w !sudo tee %

步骤5:转到主目录并编辑.bash_profile文件

cd ~

然后

nvim .bash_profile

并添加

export PATH="/usr/local/mysql/bin:$PATH"

到文件然后保存

步骤6:相对于Macintosh HD找到路径并将其添加

cd /private/etc/

然后

nvim paths

并添加

/usr/local/mysql/bin

*您会再次注意到该文件具有只读访问权限,因此,如果您使用vim或neovim

:w !sudo tee % 

然后

cd ~

然后通过运行您的更改刷新终端

source .bash_profile

最后

pip3 install mysqlclient

还有中提琴 记住这是一种氛围。

I had been debugging this problem forever – 3 hours 17 mins. What particularly annoyed me was that I already had sql installed on my system through prior uni work but pip/pip3 wasn’t recognising it. These threads above and many other I scoured the internet for were helpful in eluminating the problem but didn’t actually solve things.

ANSWER

Pip is looking for mysql binaries in the Homebrew Directory which is located relative to Macintosh HD @

/usr/local/Cellar/

so I found that this requires you making a few changes

step 1: Download MySql if not already done so https://dev.mysql.com/downloads/

Step 2: Locate it relative to Macintosh HD and cd

/usr/local/mysql/bin

Step 3: Once there open terminal and use a text editor of choice – I’m a neovim guy myself so I typed (doesn’t automatically come with Mac… another story for another day)

nvim mysql_config

Step 4: You will see at approx line 112

# Create options 
libs="-L$pkglibdir"
libs="$libs -l "

Change to

# Create options 
libs="-L$pkglibdir"
libs="$libs -lmysqlclient -lssl -lcrypto"

*you’ll notice that this file has read-only access so if your using vim or neovim

:w !sudo tee %

Step 5: Head to the home directory and edit the .bash_profile file

cd ~

Then

nvim .bash_profile

and add

export PATH="/usr/local/mysql/bin:$PATH"

to the file then save

Step 6: relative to Macintosh HD locate paths and add to it

cd /private/etc/

then

nvim paths

and add

/usr/local/mysql/bin

*you’ll again notice that this file has read-only access so if your using vim or neovim

:w !sudo tee % 

then

cd ~

then refresh the terminal with your changes by running

source .bash_profile

Finally

pip3 install mysqlclient

And Viola. Remember it’s a vibe.


回答 2

如果您不想安装完整的mysql,我们可以通过安装mysqlclient来解决此问题。cmd brew install mysqlclient 完成后,将要求在下面添加以下行~/.bash_profile

echo 'export PATH="/usr/local/opt/mysql-client/bin:$PATH"' >> ~/.bash_profile

关闭终端并启动新终端,然后继续 pip install mysqlclient

If you don’t want to install full mysql, we can fix this by just installing mysqlclient brew install mysqlclient Once cmd is completed it will ask to add below line to ~/.bash_profile:

echo 'export PATH="/usr/local/opt/mysql-client/bin:$PATH"' >> ~/.bash_profile

Close terminal and start new terminal and proceed with pip install mysqlclient


回答 3

我在MacOS Catalina上运行Python 3.6。我的问题是我尝试安装mysqlclient==1.4.2.post1,并不断抛出mysql_config not found错误。

这是我解决问题的步骤。

  1. 使用brew安装mysql-connector-c(如果已经有mysql,请先安装unlink brew unlink mysql)-brew install mysql-connector-c
  2. 打开mysql_config并编辑第112行附近的文件
# Create options 
libs="-L$pkglibdir"
libs="$libs -lmysqlclient -lssl -lcrypto"
  1. brew info openssl -这将为您提供更多有关将openssl放入PATH中需要执行的操作的信息
  2. 关于步骤3,您需要执行以下操作将openssl放入PATH中- echo 'export PATH="/usr/local/opt/openssl/bin:$PATH"' >> ~/.bash_profile
  3. 供编译器找到openssl- export LDFLAGS="-L/usr/local/opt/openssl/lib"
  4. 供编译器找到openssl- export CPPFLAGS="-I/usr/local/opt/openssl/include"

I am running Python 3.6 on MacOS Catalina. My issue was that I tried to install mysqlclient==1.4.2.post1 and it keeps throwing mysql_config not found error.

This is the steps I took to solve the issue.

  1. Install mysql-connector-c using brew (if you have mysql already install unlink first brew unlink mysql) – brew install mysql-connector-c
  2. Open mysql_config and edit the file around line 112
# Create options 
libs="-L$pkglibdir"
libs="$libs -lmysqlclient -lssl -lcrypto"
  1. brew info openssl – this will give you more information on what needs to be done about putting openssl in PATH
  2. in relation to step 3, you need to do this to put openssl in PATH – echo 'export PATH="/usr/local/opt/openssl/bin:$PATH"' >> ~/.bash_profile
  3. for compilers to find openssl – export LDFLAGS="-L/usr/local/opt/openssl/lib"
  4. for compilers to find openssl – export CPPFLAGS="-I/usr/local/opt/openssl/include"

回答 4

当我安装mysqlclient时也会发生这种情况,

$ pip install mysqlclient

如用户3429036所说,

$ brew install mysql

Also this happens when I was installing mysqlclient,

$ pip install mysqlclient

As user3429036 said,

$ brew install mysql

回答 5

此答案适用于brew不是从官方安装而是从官方安装的MacOS用户.dmg/.pkg。该安装程序无法编辑您的PATH,从而导致出现问题:

  1. 所有MySQL命令一样mysqlmysqladminmysql_config,等不能被发现,并作为一个结果:
  2. “MySQL的预置面板”未出现在系统偏好设置,
  3. 您不能安装任何与MySQL通信的API,包括 mysqlclient

您要做的是附加MySQL bin文件夹(通常/usr/local/mysql/bin在您PATH~/.bash_profile文件中添加以下行:

export PATH="/usr/local/mysql/bin/:$PATH"

然后,您应该重新加载您~/.bash_profile的更改,以使更改在当前的终端会话中生效:

source ~/.bash_profile

mysqlclient但是,在安装之前,您需要接受XcodeBuild许可证:

sudo xcodebuild -license

按照他们的指示退出您的家人,之后您应该可以顺利安装mysqlclient

pip install mysqlclient

安装在此之后,你必须做一件事来解决运行时错误,船舶与MySQL(动态库libmysqlclient.dylib未找到),加入这行到你的系统动态库路径:

export DYLD_LIBRARY_PATH=/usr/local/mysql/lib/:$DYLD_LIBRARY_PATH

This answer is for MacOS users who did not install from brew but rather from the official .dmg/.pkg. That installer fails to edit your PATH, causing things to break out of the box:

  1. All MySQL commands like mysql, mysqladmin, mysql_config, etc cannot be found, and as a result:
  2. the “MySQL Preference Pane” fails to appear in System Preferences, and
  3. you cannot install any API that communicates with MySQL, including mysqlclient

What you have to do is appending the MySQL bin folder (typically /usr/local/mysql/bin in your PATH by adding this line in your ~/.bash_profile file:

export PATH="/usr/local/mysql/bin/:$PATH"

You should then reload your ~/.bash_profile for the change to take effect in your current Terminal session:

source ~/.bash_profile

Before installing mysqlclient, however, you need to accept the XcodeBuild license:

sudo xcodebuild -license

Follow their directions to sign away your family, after which you should be able to install mysqlclient without issue:

pip install mysqlclient

After installing that, you must do one more thing to fix a runtime bug that ships with MySQL (Dynamic Library libmysqlclient.dylib not found), by adding this line to your system dynamic libraries path:

export DYLD_LIBRARY_PATH=/usr/local/mysql/lib/:$DYLD_LIBRARY_PATH


回答 6

如果您通过指定版本使用Homebrew安装了mysql ,则mysql_config将出现在此处。– /usr/local/Cellar/mysql@5.6/5.6.47/bin

您可以在/ usr / local /目录中使用ls命令找到sql bin的路径

/usr/local/Cellar/mysql@5.6/5.6.47/bin

像这样将路径添加到bash配置文件。

nano ~/.bash_profile

export PATH="/usr/local/Cellar/mysql@5.6/5.6.47/bin:$PATH"

If you have installed mysql using Homebrew by specifying a version then mysql_config would be present here. – /usr/local/Cellar/mysql@5.6/5.6.47/bin

you can find the path of the sql bin by using ls command in /usr/local/ directory

/usr/local/Cellar/mysql@5.6/5.6.47/bin

Add the path to bash profile like this.

nano ~/.bash_profile

export PATH="/usr/local/Cellar/mysql@5.6/5.6.47/bin:$PATH"

回答 7

在我的情况下,问题是我在python虚拟环境中运行命令,尽管我将其放在.bash_profile文件中,但它没有/ usr / local / mysql / bin的路径。只是在虚拟环境中导出路径对我有用。

对于您的信息,sql_config位于bin目录中。

The problem in my case was that I was running the command inside a python virtual environment and it didn’t had the path to /usr/local/mysql/bin though I have put it in the .bash_profile file. Just exporting the path in the virtual env worked for me.

For your info sql_config resides inside bin directory.


回答 8

对我来说,安装brew或apt-get也不容易,所以我通过以下网址下载了mysql:https//dev.mysql.com/downloads/connector/python/,并安装了它。因此,我可以在以下目录中找到mysql_config:/ usr / local / mysql / bin

下一步是:

  1. 导出PATH = $ PATH:/ usr / local / mysql / bin
  2. 点安装MySQL-python == 1.2.5

Install brew or apt-get is also not easy for me so I downloaded mysql via: https://dev.mysql.com/downloads/connector/python/, installed it. So I can find mysql_config int this directory: /usr/local/mysql/bin

the next step is:

  1. export PATH=$PATH:/usr/local/mysql/bin
  2. pip install MySQL-python==1.2.5

如何将SQL查询结果转换为PANDAS数据结构?

问题:如何将SQL查询结果转换为PANDAS数据结构?

在这个问题上的任何帮助将不胜感激。

因此,基本上我想对我的SQL数据库运行查询并将返回的数据存储为Pandas数据结构。

我已附上查询代码。

我正在阅读有关Pandas的文档,但是在识别查询的返回类型时遇到了问题。

我试图打印查询结果,但没有提供任何有用的信息。

谢谢!!!!

from sqlalchemy import create_engine

engine2 = create_engine('mysql://THE DATABASE I AM ACCESSING')
connection2 = engine2.connect()
dataid = 1022
resoverall = connection2.execute("
  SELECT 
      sum(BLABLA) AS BLA,
      sum(BLABLABLA2) AS BLABLABLA2,
      sum(SOME_INT) AS SOME_INT,
      sum(SOME_INT2) AS SOME_INT2,
      100*sum(SOME_INT2)/sum(SOME_INT) AS ctr,
      sum(SOME_INT2)/sum(SOME_INT) AS cpc
   FROM daily_report_cooked
   WHERE campaign_id = '%s'", %dataid)

因此,我有点想了解变量“ resoverall”的格式/数据类型是什么,以及如何将其与PANDAS数据结构一起使用。

Any help on this problem will be greatly appreciated.

So basically I want to run a query to my SQL database and store the returned data as Pandas data structure.

I have attached code for query.

I am reading the documentation on Pandas, but I have problem to identify the return type of my query.

I tried to print the query result, but it doesn’t give any useful information.

Thanks!!!!

from sqlalchemy import create_engine

engine2 = create_engine('mysql://THE DATABASE I AM ACCESSING')
connection2 = engine2.connect()
dataid = 1022
resoverall = connection2.execute("
  SELECT 
      sum(BLABLA) AS BLA,
      sum(BLABLABLA2) AS BLABLABLA2,
      sum(SOME_INT) AS SOME_INT,
      sum(SOME_INT2) AS SOME_INT2,
      100*sum(SOME_INT2)/sum(SOME_INT) AS ctr,
      sum(SOME_INT2)/sum(SOME_INT) AS cpc
   FROM daily_report_cooked
   WHERE campaign_id = '%s'", %dataid)

So I sort of want to understand what’s the format/datatype of my variable “resoverall” and how to put it with PANDAS data structure.


回答 0

这是完成任务的最短代码:

from pandas import DataFrame
df = DataFrame(resoverall.fetchall())
df.columns = resoverall.keys()

您可以像Paul的回答中所说的那样幻想和分析类型。

Here’s the shortest code that will do the job:

from pandas import DataFrame
df = DataFrame(resoverall.fetchall())
df.columns = resoverall.keys()

You can go fancier and parse the types as in Paul’s answer.


回答 1

编辑:2015年3月

如下所述,熊猫现在使用SQLAlchemy读取(read_sql)并将其插入(to_sql)数据库。以下应该工作

import pandas as pd

df = pd.read_sql(sql, cnxn)

以前的答案: 通过类似问题的麦克贝克斯

import pyodbc
import pandas.io.sql as psql

cnxn = pyodbc.connect(connection_info) 
cursor = cnxn.cursor()
sql = "SELECT * FROM TABLE"

df = psql.frame_query(sql, cnxn)
cnxn.close()

Edit: Mar. 2015

As noted below, pandas now uses SQLAlchemy to both read from (read_sql) and insert into (to_sql) a database. The following should work

import pandas as pd

df = pd.read_sql(sql, cnxn)

Previous answer: Via mikebmassey from a similar question

import pyodbc
import pandas.io.sql as psql

cnxn = pyodbc.connect(connection_info) 
cursor = cnxn.cursor()
sql = "SELECT * FROM TABLE"

df = psql.frame_query(sql, cnxn)
cnxn.close()

回答 2

如果您使用的是SQLAlchemy的ORM而不是表达式语言,则可能会发现自己想要将类型的对象转换sqlalchemy.orm.query.Query为Pandas数据框。

最干净的方法是从查询的statement属性获取生成的SQL,然后使用pandas的read_sql()方法执行它。例如,从名为的查询对象开始query

df = pd.read_sql(query.statement, query.session.bind)

If you are using SQLAlchemy’s ORM rather than the expression language, you might find yourself wanting to convert an object of type sqlalchemy.orm.query.Query to a Pandas data frame.

The cleanest approach is to get the generated SQL from the query’s statement attribute, and then execute it with pandas’s read_sql() method. E.g., starting with a Query object called query:

df = pd.read_sql(query.statement, query.session.bind)

回答 3

编辑2014-09-30:

熊猫现在具有read_sql功能。您肯定要使用它。

原始答案:

我无法使用SQLAlchemy帮助您-我总是根据需要使用pyodbc,MySQLdb或psychopg2。但是这样做的时候,像下面这样一个简单的函数往往可以满足我的需求:

import decimal

import pydobc
import numpy as np
import pandas

cnn, cur = myConnectToDBfunction()
cmd = "SELECT * FROM myTable"
cur.execute(cmd)
dataframe = __processCursor(cur, dataframe=True)

def __processCursor(cur, dataframe=False, index=None):
    '''
    Processes a database cursor with data on it into either
    a structured numpy array or a pandas dataframe.

    input:
    cur - a pyodbc cursor that has just received data
    dataframe - bool. if false, a numpy record array is returned
                if true, return a pandas dataframe
    index - list of column(s) to use as index in a pandas dataframe
    '''
    datatypes = []
    colinfo = cur.description
    for col in colinfo:
        if col[1] == unicode:
            datatypes.append((col[0], 'U%d' % col[3]))
        elif col[1] == str:
            datatypes.append((col[0], 'S%d' % col[3]))
        elif col[1] in [float, decimal.Decimal]:
            datatypes.append((col[0], 'f4'))
        elif col[1] == datetime.datetime:
            datatypes.append((col[0], 'O4'))
        elif col[1] == int:
            datatypes.append((col[0], 'i4'))

    data = []
    for row in cur:
        data.append(tuple(row))

    array = np.array(data, dtype=datatypes)
    if dataframe:
        output = pandas.DataFrame.from_records(array)

        if index is not None:
            output = output.set_index(index)

    else:
        output = array

    return output

Edit 2014-09-30:

pandas now has a read_sql function. You definitely want to use that instead.

Original answer:

I can’t help you with SQLAlchemy — I always use pyodbc, MySQLdb, or psychopg2 as needed. But when doing so, a function as simple as the one below tends to suit my needs:

import decimal

import pydobc
import numpy as np
import pandas

cnn, cur = myConnectToDBfunction()
cmd = "SELECT * FROM myTable"
cur.execute(cmd)
dataframe = __processCursor(cur, dataframe=True)

def __processCursor(cur, dataframe=False, index=None):
    '''
    Processes a database cursor with data on it into either
    a structured numpy array or a pandas dataframe.

    input:
    cur - a pyodbc cursor that has just received data
    dataframe - bool. if false, a numpy record array is returned
                if true, return a pandas dataframe
    index - list of column(s) to use as index in a pandas dataframe
    '''
    datatypes = []
    colinfo = cur.description
    for col in colinfo:
        if col[1] == unicode:
            datatypes.append((col[0], 'U%d' % col[3]))
        elif col[1] == str:
            datatypes.append((col[0], 'S%d' % col[3]))
        elif col[1] in [float, decimal.Decimal]:
            datatypes.append((col[0], 'f4'))
        elif col[1] == datetime.datetime:
            datatypes.append((col[0], 'O4'))
        elif col[1] == int:
            datatypes.append((col[0], 'i4'))

    data = []
    for row in cur:
        data.append(tuple(row))

    array = np.array(data, dtype=datatypes)
    if dataframe:
        output = pandas.DataFrame.from_records(array)

        if index is not None:
            output = output.set_index(index)

    else:
        output = array

    return output

回答 4

MySQL连接器

对于使用mysql连接器的用户,可以将此代码作为开始。(感谢@Daniel Velkov)

二手裁判:


import pandas as pd
import mysql.connector

# Setup MySQL connection
db = mysql.connector.connect(
    host="<IP>",              # your host, usually localhost
    user="<USER>",            # your username
    password="<PASS>",        # your password
    database="<DATABASE>"     # name of the data base
)   

# You must create a Cursor object. It will let you execute all the queries you need
cur = db.cursor()

# Use all the SQL you like
cur.execute("SELECT * FROM <TABLE>")

# Put it all to a data frame
sql_data = pd.DataFrame(cur.fetchall())
sql_data.columns = cur.column_names

# Close the session
db.close()

# Show the data
print(sql_data.head())

MySQL Connector

For those that works with the mysql connector you can use this code as a start. (Thanks to @Daniel Velkov)

Used refs:


import pandas as pd
import mysql.connector

# Setup MySQL connection
db = mysql.connector.connect(
    host="<IP>",              # your host, usually localhost
    user="<USER>",            # your username
    password="<PASS>",        # your password
    database="<DATABASE>"     # name of the data base
)   

# You must create a Cursor object. It will let you execute all the queries you need
cur = db.cursor()

# Use all the SQL you like
cur.execute("SELECT * FROM <TABLE>")

# Put it all to a data frame
sql_data = pd.DataFrame(cur.fetchall())
sql_data.columns = cur.column_names

# Close the session
db.close()

# Show the data
print(sql_data.head())

回答 5

这是我使用的代码。希望这可以帮助。

import pandas as pd
from sqlalchemy import create_engine

def getData():
  # Parameters
  ServerName = "my_server"
  Database = "my_db"
  UserPwd = "user:pwd"
  Driver = "driver=SQL Server Native Client 11.0"

  # Create the connection
  engine = create_engine('mssql+pyodbc://' + UserPwd + '@' + ServerName + '/' + Database + "?" + Driver)

  sql = "select * from mytable"
  df = pd.read_sql(sql, engine)
  return df

df2 = getData()
print(df2)

Here’s the code I use. Hope this helps.

import pandas as pd
from sqlalchemy import create_engine

def getData():
  # Parameters
  ServerName = "my_server"
  Database = "my_db"
  UserPwd = "user:pwd"
  Driver = "driver=SQL Server Native Client 11.0"

  # Create the connection
  engine = create_engine('mssql+pyodbc://' + UserPwd + '@' + ServerName + '/' + Database + "?" + Driver)

  sql = "select * from mytable"
  df = pd.read_sql(sql, engine)
  return df

df2 = getData()
print(df2)

回答 6

这是对您的问题的简短回答:

from __future__ import print_function
import MySQLdb
import numpy as np
import pandas as pd
import xlrd

# Connecting to MySQL Database
connection = MySQLdb.connect(
             host="hostname",
             port=0000,
             user="userID",
             passwd="password",
             db="table_documents",
             charset='utf8'
           )
print(connection)
#getting data from database into a dataframe
sql_for_df = 'select * from tabledata'
df_from_database = pd.read_sql(sql_for_df , connection)

This is a short and crisp answer to your problem:

from __future__ import print_function
import MySQLdb
import numpy as np
import pandas as pd
import xlrd

# Connecting to MySQL Database
connection = MySQLdb.connect(
             host="hostname",
             port=0000,
             user="userID",
             passwd="password",
             db="table_documents",
             charset='utf8'
           )
print(connection)
#getting data from database into a dataframe
sql_for_df = 'select * from tabledata'
df_from_database = pd.read_sql(sql_for_df , connection)

回答 7

1.使用MySQL-connector-python

# pip install mysql-connector-python

import mysql.connector
import pandas as pd

mydb = mysql.connector.connect(
    host = 'host',
    user = 'username',
    passwd = 'pass',
    database = 'db_name'
)
query = 'select * from table_name'
df = pd.read_sql(query, con = mydb)
print(df)

2.使用SQLAlchemy

# pip install pymysql
# pip install sqlalchemy

import pandas as pd
import sqlalchemy

engine = sqlalchemy.create_engine('mysql+pymysql://username:password@localhost:3306/db_name')

query = '''
select * from table_name
'''
df = pd.read_sql_query(query, engine)
print(df)

1. Using MySQL-connector-python

# pip install mysql-connector-python

import mysql.connector
import pandas as pd

mydb = mysql.connector.connect(
    host = 'host',
    user = 'username',
    passwd = 'pass',
    database = 'db_name'
)
query = 'select * from table_name'
df = pd.read_sql(query, con = mydb)
print(df)

2. Using SQLAlchemy

# pip install pymysql
# pip install sqlalchemy

import pandas as pd
import sqlalchemy

engine = sqlalchemy.create_engine('mysql+pymysql://username:password@localhost:3306/db_name')

query = '''
select * from table_name
'''
df = pd.read_sql_query(query, engine)
print(df)

回答 8

像Nathan一样,我经常想将sqlalchemy或sqlsoup Query的结果转储到Pandas数据框中。我自己的解决方案是:

query = session.query(tbl.Field1, tbl.Field2)
DataFrame(query.all(), columns=[column['name'] for column in query.column_descriptions])

Like Nathan, I often want to dump the results of a sqlalchemy or sqlsoup Query into a Pandas data frame. My own solution for this is:

query = session.query(tbl.Field1, tbl.Field2)
DataFrame(query.all(), columns=[column['name'] for column in query.column_descriptions])

回答 9

resoverall是sqlalchemy ResultProxy对象。您可以在sqlalchemy文档中阅读有关它的更多信息,后者介绍了使用Engines and Connections的基本用法。这里重要的resoverall是dict之类的。

熊猫喜欢像dict这样的对象来创建其数据结构,请参见 在线文档

祝您好运sqlalchemy和熊猫。

resoverall is a sqlalchemy ResultProxy object. You can read more about it in the sqlalchemy docs, the latter explains basic usage of working with Engines and Connections. Important here is that resoverall is dict like.

Pandas likes dict like objects to create its data structures, see the online docs

Good luck with sqlalchemy and pandas.


回答 10

简单地使用pandaspyodbc在一起。您必须connstr根据数据库规范修改连接字符串()。

import pyodbc
import pandas as pd

# MSSQL Connection String Example
connstr = "Server=myServerAddress;Database=myDB;User Id=myUsername;Password=myPass;"

# Query Database and Create DataFrame Using Results
df = pd.read_sql("select * from myTable", pyodbc.connect(connstr))

我已经使用pyodbc了多个企业数据库(例如SQL Server,MySQL,MariaDB,IBM)。

Simply use pandas and pyodbc together. You’ll have to modify your connection string (connstr) according to your database specifications.

import pyodbc
import pandas as pd

# MSSQL Connection String Example
connstr = "Server=myServerAddress;Database=myDB;User Id=myUsername;Password=myPass;"

# Query Database and Create DataFrame Using Results
df = pd.read_sql("select * from myTable", pyodbc.connect(connstr))

I’ve used pyodbc with several enterprise databases (e.g. SQL Server, MySQL, MariaDB, IBM).


回答 11

这个问题很旧,但是我想加两分钱。我读到的问题是“我想对我的[my] SQL数据库运行查询并将返回的数据存储为Pandas数据结构[DataFrame]。”

从代码中看起来您的意思是mysql数据库,并假设您的意思是pandas DataFrame。

import MySQLdb as mdb
import pandas.io.sql as sql
from pandas import *

conn = mdb.connect('<server>','<user>','<pass>','<db>');
df = sql.read_frame('<query>', conn)

例如,

conn = mdb.connect('localhost','myname','mypass','testdb');
df = sql.read_frame('select * from testTable', conn)

这会将testTable的所有行导入到DataFrame中。

This question is old, but I wanted to add my two-cents. I read the question as ” I want to run a query to my [my]SQL database and store the returned data as Pandas data structure [DataFrame].”

From the code it looks like you mean mysql database and assume you mean pandas DataFrame.

import MySQLdb as mdb
import pandas.io.sql as sql
from pandas import *

conn = mdb.connect('<server>','<user>','<pass>','<db>');
df = sql.read_frame('<query>', conn)

For example,

conn = mdb.connect('localhost','myname','mypass','testdb');
df = sql.read_frame('select * from testTable', conn)

This will import all rows of testTable into a DataFrame.


回答 12

这是我的。以防万一,如果您使用“ pymysql”:

import pymysql
from pandas import DataFrame

host   = 'localhost'
port   = 3306
user   = 'yourUserName'
passwd = 'yourPassword'
db     = 'yourDatabase'

cnx    = pymysql.connect(host=host, port=port, user=user, passwd=passwd, db=db)
cur    = cnx.cursor()

query  = """ SELECT * FROM yourTable LIMIT 10"""
cur.execute(query)

field_names = [i[0] for i in cur.description]
get_data = [xx for xx in cur]

cur.close()
cnx.close()

df = DataFrame(get_data)
df.columns = field_names

Here is mine. Just in case if you are using “pymysql”:

import pymysql
from pandas import DataFrame

host   = 'localhost'
port   = 3306
user   = 'yourUserName'
passwd = 'yourPassword'
db     = 'yourDatabase'

cnx    = pymysql.connect(host=host, port=port, user=user, passwd=passwd, db=db)
cur    = cnx.cursor()

query  = """ SELECT * FROM yourTable LIMIT 10"""
cur.execute(query)

field_names = [i[0] for i in cur.description]
get_data = [xx for xx in cur]

cur.close()
cnx.close()

df = DataFrame(get_data)
df.columns = field_names

回答 13

pandas.io.sql.write_frame已弃用。 https://pandas.pydata.org/pandas-docs/version/0.15.2/generated/pandas.io.sql.write_frame.html

应该更改为使用pandas.DataFrame.to_sql https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.to_sql.html

还有另一种解决方案。 PYODBC到Pandas-DataFrame不起作用-传递的值的形状为(x,y),索引表示为(w,z)

从熊猫0.12(我相信)开始,您可以:

import pandas
import pyodbc

sql = 'select * from table'
cnn = pyodbc.connect(...)

data = pandas.read_sql(sql, cnn)

在0.12之前,您可以执行以下操作:

import pandas
from pandas.io.sql import read_frame
import pyodbc

sql = 'select * from table'
cnn = pyodbc.connect(...)

data = read_frame(sql, cnn)

pandas.io.sql.write_frame is DEPRECATED. https://pandas.pydata.org/pandas-docs/version/0.15.2/generated/pandas.io.sql.write_frame.html

Should change to use pandas.DataFrame.to_sql https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.to_sql.html

There is another solution. PYODBC to Pandas – DataFrame not working – Shape of passed values is (x,y), indices imply (w,z)

As of Pandas 0.12 (I believe) you can do:

import pandas
import pyodbc

sql = 'select * from table'
cnn = pyodbc.connect(...)

data = pandas.read_sql(sql, cnn)

Prior to 0.12, you could do:

import pandas
from pandas.io.sql import read_frame
import pyodbc

sql = 'select * from table'
cnn = pyodbc.connect(...)

data = read_frame(sql, cnn)

回答 14

离上一篇帖子很久了,但也许可以帮助某人…

比Paul H更短的路:

my_dic = session.query(query.all())
my_df = pandas.DataFrame.from_dict(my_dic)

Long time from last post but maybe it helps someone…

Shorted way than Paul H:

my_dic = session.query(query.all())
my_df = pandas.DataFrame.from_dict(my_dic)

回答 15

我这样做的最好方法

db.execute(query) where db=db_class() #database class
    mydata=[x for x in db.fetchall()]
    df=pd.DataFrame(data=mydata)

best way I do this

db.execute(query) where db=db_class() #database class
    mydata=[x for x in db.fetchall()]
    df=pd.DataFrame(data=mydata)

回答 16

如果结果类型为ResultSet,则应首先将其转换为字典。然后,将自动收集DataFrame列

这适用于我的情况:

df = pd.DataFrame([dict(r) for r in resoverall])

If the result type is ResultSet, you should convert it to dictionary first. Then the DataFrame columns will be collected automatically.

This works on my case:

df = pd.DataFrame([dict(r) for r in resoverall])

SQLAlchemy-获取表列表

问题:SQLAlchemy-获取表列表

我在文档中找不到与此有关的任何信息,但是如何获得在SQLAlchemy中创建的表的列表?

我使用了类方法来创建表。

I couldn’t find any information about this in the documentation, but how can I get a list of tables created in SQLAlchemy?

I used the class method to create the tables.


回答 0

所有表都收集在tablesSQLAlchemy MetaData对象的属性中。要获取这些表的名称列表:

>>> metadata.tables.keys()
['posts', 'comments', 'users']

如果使用声明性扩展,则可能不是您自己管理元数据。幸运的是,元数据仍然存在于基类中,

>>> Base = sqlalchemy.ext.declarative.declarative_base()
>>> Base.metadata
MetaData(None)

如果您试图弄清楚数据库中存在哪些表,甚至还没有告诉SQLAlchemy的表,那么可以使用表反射。然后,SQLAlchemy将检查数据库并使用所有缺少的表更新元数据。

>>> metadata.reflect(engine)

对于Postgres,如果您有多个架构,则需要遍历引擎中的所有架构:

from sqlalchemy import inspect
inspector = inspect(engine)
schemas = inspector.get_schema_names()

for schema in schemas:
    print("schema: %s" % schema)
    for table_name in inspector.get_table_names(schema=schema):
        for column in inspector.get_columns(table_name, schema=schema):
            print("Column: %s" % column)

All of the tables are collected in the tables attribute of the SQLAlchemy MetaData object. To get a list of the names of those tables:

>>> metadata.tables.keys()
['posts', 'comments', 'users']

If you’re using the declarative extension, then you probably aren’t managing the metadata yourself. Fortunately, the metadata is still present on the baseclass,

>>> Base = sqlalchemy.ext.declarative.declarative_base()
>>> Base.metadata
MetaData(None)

If you are trying to figure out what tables are present in your database, even among the ones you haven’t even told SQLAlchemy about yet, then you can use table reflection. SQLAlchemy will then inspect the database and update the metadata with all of the missing tables.

>>> metadata.reflect(engine)

For Postgres, if you have multiple schemas, you’ll need to loop thru all the schemas in the engine:

from sqlalchemy import inspect
inspector = inspect(engine)
schemas = inspector.get_schema_names()

for schema in schemas:
    print("schema: %s" % schema)
    for table_name in inspector.get_table_names(schema=schema):
        for column in inspector.get_columns(table_name, schema=schema):
            print("Column: %s" % column)

回答 1

engine对象中有一个方法来获取表名称列表。engine.table_names()

There is a method in engine object to fetch the list of tables name. engine.table_names()


回答 2

from sqlalchemy import create_engine
engine = create_engine('postgresql://use:pass@localhost/DBname')
print (engine.table_names())
from sqlalchemy import create_engine
engine = create_engine('postgresql://use:pass@localhost/DBname')
print (engine.table_names())

回答 3

在python解释器中,使用db.engine.table_names()

$ python
>>> from myapp import db
>>> db.engine.table_names()

Within the python interpreter use db.engine.table_names()

$ python
>>> from myapp import db
>>> db.engine.table_names()

回答 4

我一直在寻找这样的东西:

from sqlalchemy import create_engine
eng = create_engine('mysql+pymysql://root:password@localhost:3306', pool_recycle=3600)
q = eng.execute('SHOW TABLES')

available_tables = q.fetchall()

它执行并返回所有表。

更新:

Postgres:

eng = create_engine('postgresql+psycopg2://root:password@localhost/
q = eng.execute('SELECT * FROM pg_catalog.pg_tables')

I was looking for something like this:

from sqlalchemy import create_engine
eng = create_engine('mysql+pymysql://root:password@localhost:3306', pool_recycle=3600)
q = eng.execute('SHOW TABLES')

available_tables = q.fetchall()

It does an execute and returns all of the tables.

update:

Postgres:

eng = create_engine('postgresql+psycopg2://root:password@localhost/
q = eng.execute('SELECT * FROM pg_catalog.pg_tables')

回答 5

用于创建表的元数据对象在字典中具有该对象。

metadata.tables.keys()

The metadata object that you created the tables with has that in a dictionary.

metadata.tables.keys()

回答 6

我正在解决相同的问题,并找到了这篇文章。经过一些尝试运行后,我建议使用下面的方法列出所有表:(由zerocog提及)

metadata = MetaData()
metadata.reflect(bind=engine)
for table in metadata.sorted_tables:
    print(table)

这对于直接处理表很有用,我建议您这样做。

并使用以下代码获取表名:

for table_name in engine.table_names():
    print(table_name)

“ metadata.tables”为表名和表对象提供了一个Dict。这对于快速查询也很有用。

I’m solving same problem and found this post. After some try run, I would suggest use below to list all tables: (mentioned by zerocog)

metadata = MetaData()
metadata.reflect(bind=engine)
for table in metadata.sorted_tables:
    print(table)

This is useful for direct table handling and I feel is recommended.

And use below code to get table names:

for table_name in engine.table_names():
    print(table_name)

“metadata.tables” provides a Dict for table name and Table object. which would also be useful for quick query.


回答 7

一次反映所有表允许您也检索隐藏的表名。我创建了一些临时表,他们出现了

meta = MetaData()
meta.reflect(bind=myengine)
for table in reversed(meta.sorted_tables):
    print table

参考http://docs.sqlalchemy.org/en/latest/core/reflection.html

Reflecting All Tables at Once allows you to retrieve hidden table names too. I created some temporary tables and they showed up with

meta = MetaData()
meta.reflect(bind=myengine)
for table in reversed(meta.sorted_tables):
    print table

Reference http://docs.sqlalchemy.org/en/latest/core/reflection.html


回答 8

就这么简单:

engine.table_names()

另外,要测试表是否存在:

engine.has_table(table_name)

Just this simple:

engine.table_names()

Also, to test whether a table exists:

engine.has_table(table_name)