标签归档:sqlalchemy

SQLAlchemy默认DateTime

问题:SQLAlchemy默认DateTime

这是我的声明性模型:

import datetime
from sqlalchemy import Column, Integer, DateTime
from sqlalchemy.ext.declarative import declarative_base

Base = declarative_base()

class Test(Base):
    __tablename__ = 'test'

    id = Column(Integer, primary_key=True)
    created_date = DateTime(default=datetime.datetime.utcnow)

但是,当我尝试导入此模块时,出现此错误:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "orm/models2.py", line 37, in <module>
    class Test(Base):
  File "orm/models2.py", line 41, in Test
    created_date = sqlalchemy.DateTime(default=datetime.datetime.utcnow)
TypeError: __init__() got an unexpected keyword argument 'default'

如果使用整数类型,则可以设置默认值。这是怎么回事?

This is my declarative model:

import datetime
from sqlalchemy import Column, Integer, DateTime
from sqlalchemy.ext.declarative import declarative_base

Base = declarative_base()

class Test(Base):
    __tablename__ = 'test'

    id = Column(Integer, primary_key=True)
    created_date = DateTime(default=datetime.datetime.utcnow)

However, when I try to import this module, I get this error:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "orm/models2.py", line 37, in <module>
    class Test(Base):
  File "orm/models2.py", line 41, in Test
    created_date = sqlalchemy.DateTime(default=datetime.datetime.utcnow)
TypeError: __init__() got an unexpected keyword argument 'default'

If I use an Integer type, I can set a default value. What’s going on?


回答 0

DateTime没有默认键作为输入。默认键应该是该Column功能的输入。试试这个:

import datetime
from sqlalchemy import Column, Integer, DateTime
from sqlalchemy.ext.declarative import declarative_base

Base = declarative_base()

class Test(Base):
    __tablename__ = 'test'

    id = Column(Integer, primary_key=True)
    created_date = Column(DateTime, default=datetime.datetime.utcnow)

DateTime doesn’t have a default key as an input. The default key should be an input to the Column function. Try this:

import datetime
from sqlalchemy import Column, Integer, DateTime
from sqlalchemy.ext.declarative import declarative_base

Base = declarative_base()

class Test(Base):
    __tablename__ = 'test'

    id = Column(Integer, primary_key=True)
    created_date = Column(DateTime, default=datetime.datetime.utcnow)

回答 1

计算数据库中的时间戳,而不是客户端中的时间戳

为了理智,您可能希望datetimes由数据库服务器而不是应用程序服务器来计算所有数据。计算应用程序中的时间戳可能会导致问题,因为网络等待时间是可变的,客户端会经历略微不同的时钟漂移,并且不同的编程语言有时会略有不同地计算时间。

SQLAlchemy允许您通过传递func.now()func.current_timestamp()(它们是彼此的别名)来执行此操作,该命令告诉DB计算时间戳本身。

使用SQLALchemy的 server_default

另外,对于已经告诉数据库计算值的默认值,通常最好使用server_default代替default。这告诉SQLAlchemy将默认值作为CREATE TABLE语句的一部分传递。

例如,如果您针对该表编写了一个临时脚本,则使用server_default意味着您无需担心手动向脚本添加时间戳调用-数据库将自动对其进行设置。

了解SQLAlchemy的onupdate/server_onupdate

SQLAlchemy还支持,onupdate以便每当更新该行时,它都会插入一个新的时间戳。再一次,最好告诉数据库来计算时间戳本身:

from sqlalchemy.sql import func

time_created = Column(DateTime(timezone=True), server_default=func.now())
time_updated = Column(DateTime(timezone=True), onupdate=func.now())

有一个server_onupdate参数,但与不同server_default,它实际上未在服务器端设置任何参数。它只是告诉SQLalchemy更新发生时(也许您在列上创建了触发器),数据库将更改列,因此SQLAlchemy将要求返回值,以便它可以更新相应的对象。

另一个潜在的陷阱:

您可能会惊讶地发现,如果在单个事务中进行大量更改,则它们都具有相同的时间戳。这是因为SQL标准指定CURRENT_TIMESTAMP根据事务的开始返回值。

PostgreSQL提供了非SQL标准statement_timestamp()clock_timestamp()并且在事务中更改。此处的文档:https : //www.postgresql.org/docs/current/static/functions-datetime.html#FUNCTIONS-DATETIME-CURRENT

UTC时间戳

如果要使用UTC时间戳,请func.utcnow()SQLAlchemy文档中提供的实现存根。但是,您需要自己提供适当的特定于驱动程序的功能。

Calculate timestamps within your DB, not your client

For sanity, you probably want to have all datetimes calculated by your DB server, rather than the application server. Calculating the timestamp in the application can lead to problems because network latency is variable, clients experience slightly different clock drift, and different programming languages occasionally calculate time slightly differently.

SQLAlchemy allows you to do this by passing func.now() or func.current_timestamp() (they are aliases of each other) which tells the DB to calculate the timestamp itself.

Use SQLALchemy’s server_default

Additionally, for a default where you’re already telling the DB to calculate the value, it’s generally better to use server_default instead of default. This tells SQLAlchemy to pass the default value as part of the CREATE TABLE statement.

For example, if you write an ad hoc script against this table, using server_default means you won’t need to worry about manually adding a timestamp call to your script–the database will set it automatically.

Understanding SQLAlchemy’s onupdate/server_onupdate

SQLAlchemy also supports onupdate so that anytime the row is updated it inserts a new timestamp. Again, best to tell the DB to calculate the timestamp itself:

from sqlalchemy.sql import func

time_created = Column(DateTime(timezone=True), server_default=func.now())
time_updated = Column(DateTime(timezone=True), onupdate=func.now())

There is a server_onupdate parameter, but unlike server_default, it doesn’t actually set anything serverside. It just tells SQLalchemy that your database will change the column when an update happens (perhaps you created a trigger on the column ), so SQLAlchemy will ask for the return value so it can update the corresponding object.

One other potential gotcha:

You might be surprised to notice that if you make a bunch of changes within a single transaction, they all have the same timestamp. That’s because the SQL standard specifies that CURRENT_TIMESTAMP returns values based on the start of the transaction.

PostgreSQL provides the non-SQL-standard statement_timestamp() and clock_timestamp() which do change within a transaction. Docs here: https://www.postgresql.org/docs/current/static/functions-datetime.html#FUNCTIONS-DATETIME-CURRENT

UTC timestamp

If you want to use UTC timestamps, a stub of implementation for func.utcnow() is provided in SQLAlchemy documentation. You need to provide appropriate driver-specific functions on your own though.


回答 2

您还可以默认使用sqlalchemy内置函数 DateTime

from sqlalchemy.sql import func

DT = Column(DateTime(timezone=True), default=func.now())

You can also use sqlalchemy builtin function for default DateTime

from sqlalchemy.sql import func

DT = Column(DateTime(timezone=True), default=func.now())

回答 3

您可能想要使用,onupdate=datetime.now以便UPDATE也可以更改该last_updated字段。

SQLAlchemy对于python执行的函数有两个默认值。

  • default 设置一次INSERT的值
  • onupdate还将值设置为UPDATE 上的可调用结果。

You likely want to use onupdate=datetime.now so that UPDATEs also change the last_updated field.

SQLAlchemy has two defaults for python executed functions.

  • default sets the value on INSERT, only once
  • onupdate sets the value to the callable result on UPDATE as well.

回答 4

default关键字参数应被给予Column对象。

例:

Column(u'timestamp', TIMESTAMP(timezone=True), primary_key=False, nullable=False, default=time_now),

默认值可以是可调用的,在这里我定义如下。

from pytz import timezone
from datetime import datetime

UTC = timezone('UTC')

def time_now():
    return datetime.now(UTC)

The default keyword parameter should be given to the Column object.

Example:

Column(u'timestamp', TIMESTAMP(timezone=True), primary_key=False, nullable=False, default=time_now),

The default value can be a callable, which here I defined like the following.

from pytz import timezone
from datetime import datetime

UTC = timezone('UTC')

def time_now():
    return datetime.now(UTC)

回答 5

根据PostgreSQL文档,https://www.postgresql.org/docs/9.6/static/functions-datetime.html

now, CURRENT_TIMESTAMP, LOCALTIMESTAMP return the time of transaction.

这被认为是一个功能:目的是允许单个事务具有“当前”时间的一致概念,以便同一事务内的多个修改具有相同的时间戳。

如果您不希望事务时间戳记,则可能要使用statement_timestampclock_timestamp

statement_timestamp()

返回当前语句的开始时间(更具体地说,是从客户端收到最新命令消息的时间)。statement_timestamp

clock_timestamp()

返回实际的当前时间,因此,即使在单个SQL命令中,其值也会更改。

As per PostgreSQL documentation, https://www.postgresql.org/docs/9.6/static/functions-datetime.html

now, CURRENT_TIMESTAMP, LOCALTIMESTAMP return the time of transaction.

This is considered a feature: the intent is to allow a single transaction to have a consistent notion of the “current” time, so that multiple modifications within the same transaction bear the same time stamp.

You might want to use statement_timestamp or clock_timestamp if you don’t want transaction timestamp.

statement_timestamp()

returns the start time of the current statement (more specifically, the time of receipt of the latest command message from the client). statement_timestamp

clock_timestamp()

returns the actual current time, and therefore its value changes even within a single SQL command.


sqlalchemy在多列中唯一

问题:sqlalchemy在多列中唯一

假设我有一个代表位置的类。位置“属于”客户。位置由Unicode 10个字符代码标识。对于特定客户,“位置代码”在位置之间应该是唯一的。

The two below fields in combination should be unique
customer_id = Column(Integer,ForeignKey('customers.customer_id')
location_code = Column(Unicode(10))

因此,如果我有两个客户,客户“ 123”和客户“ 456”。它们都可以有一个称为“ main”的位置,但都不能有两个称为main的位置。

我可以在业务逻辑中处理此问题,但我想确保没有办法轻松地在sqlalchemy中添加需求。unique = True选项似乎仅在应用于特定字段时才起作用,这将导致整个表仅对所有位置具有唯一代码。

Let’s say that I have a class that represents locations. Locations “belong” to customers. Locations are identified by a unicode 10 character code. The “location code” should be unique among the locations for a specific customer.

The two below fields in combination should be unique
customer_id = Column(Integer,ForeignKey('customers.customer_id')
location_code = Column(Unicode(10))

So if i have two customers, customer “123” and customer “456”. They both can have a location called “main” but neither could have two locations called main.

I can handle this in the business logic but I want to make sure there is no way to easily add the requirement in sqlalchemy. The unique=True option seems to only work when applied to a specific field and it would cause the entire table to only have a unique code for all locations.


回答 0

从以下文档的摘录Column

unique –为True时,指示此列包含唯一约束,或者,如果index也为True,则指示应使用唯一标志创建索引。要在约束/索引中指定多个列或指定一个显式名称,请显式使用 UniqueConstraintIndex构造。

由于这些属于表而不属于映射的类,因此可以在表定义中声明它们,或者如果使用声明性声明,例如__table_args__

# version1: table definition
mytable = Table('mytable', meta,
    # ...
    Column('customer_id', Integer, ForeignKey('customers.customer_id')),
    Column('location_code', Unicode(10)),

    UniqueConstraint('customer_id', 'location_code', name='uix_1')
    )
# or the index, which will ensure uniqueness as well
Index('myindex', mytable.c.customer_id, mytable.c.location_code, unique=True)


# version2: declarative
class Location(Base):
    __tablename__ = 'locations'
    id = Column(Integer, primary_key = True)
    customer_id = Column(Integer, ForeignKey('customers.customer_id'), nullable=False)
    location_code = Column(Unicode(10), nullable=False)
    __table_args__ = (UniqueConstraint('customer_id', 'location_code', name='_customer_location_uc'),
                     )

Extract from the documentation of the Column:

unique – When True, indicates that this column contains a unique constraint, or if index is True as well, indicates that the Index should be created with the unique flag. To specify multiple columns in the constraint/index or to specify an explicit name, use the UniqueConstraint or Index constructs explicitly.

As these belong to a Table and not to a mapped Class, one declares those in the table definition, or if using declarative as in the __table_args__:

# version1: table definition
mytable = Table('mytable', meta,
    # ...
    Column('customer_id', Integer, ForeignKey('customers.customer_id')),
    Column('location_code', Unicode(10)),

    UniqueConstraint('customer_id', 'location_code', name='uix_1')
    )
# or the index, which will ensure uniqueness as well
Index('myindex', mytable.c.customer_id, mytable.c.location_code, unique=True)


# version2: declarative
class Location(Base):
    __tablename__ = 'locations'
    id = Column(Integer, primary_key = True)
    customer_id = Column(Integer, ForeignKey('customers.customer_id'), nullable=False)
    location_code = Column(Unicode(10), nullable=False)
    __table_args__ = (UniqueConstraint('customer_id', 'location_code', name='_customer_location_uc'),
                     )

回答 1

from flask_sqlalchemy import SQLAlchemy
db = SQLAlchemy()

class Location(Base):
      __table_args__ = (
        # this can be db.PrimaryKeyConstraint if you want it to be a primary key
        db.UniqueConstraint('customer_id', 'location_code'))
      customer_id = Column(Integer,ForeignKey('customers.customer_id')
      location_code = Column(Unicode(10))
from flask_sqlalchemy import SQLAlchemy
db = SQLAlchemy()

class Location(Base):
      __table_args__ = (
        # this can be db.PrimaryKeyConstraint if you want it to be a primary key
        db.UniqueConstraint('customer_id', 'location_code'),
      )
      customer_id = Column(Integer,ForeignKey('customers.customer_id')
      location_code = Column(Unicode(10))

我如何知道是否可以禁用SQLALCHEMY_TRACK_MODIFICATIONS?

问题:我如何知道是否可以禁用SQLALCHEMY_TRACK_MODIFICATIONS?

每次我运行使用Flask-SQLAlchemy的应用程序时,都会收到以下警告,提示该SQLALCHEMY_TRACK_MODIFICATIONS选项将被禁用。

/home/david/.virtualenvs/flask-sqlalchemy/lib/python3.5/site-packages/flask_sqlalchemy/__init__.py:800: UserWarning: SQLALCHEMY_TRACK_MODIFICATIONS adds significant overhead and will be disabled by default in the future.  Set it to True to suppress this warning.
  warnings.warn('SQLALCHEMY_TRACK_MODIFICATIONS adds significant overhead and will be disabled by default in the future.  Set it to True to suppress this warning.')

我试图找出此选项的作用,但是Flask-SQLAlchemy文档尚不清楚该跟踪的用途。

SQLALCHEMY_TRACK_MODIFICATIONS

如果设置为True(默认值),Flask-SQLAlchemy将跟踪对象的修改并发出信号。这需要额外的内存,如果不需要,可以将其禁用。

如何确定我的项目是否需要,SQLALCHEMY_TRACK_MODIFICATIONS = True或者是否可以安全地禁用此功能并在服务器上节省内存?

Every time I run my app that uses Flask-SQLAlchemy I get the following warning that the SQLALCHEMY_TRACK_MODIFICATIONS option will be disabled.

/home/david/.virtualenvs/flask-sqlalchemy/lib/python3.5/site-packages/flask_sqlalchemy/__init__.py:800: UserWarning: SQLALCHEMY_TRACK_MODIFICATIONS adds significant overhead and will be disabled by default in the future.  Set it to True to suppress this warning.
  warnings.warn('SQLALCHEMY_TRACK_MODIFICATIONS adds significant overhead and will be disabled by default in the future.  Set it to True to suppress this warning.')

I tried to find out what this option does, but the Flask-SQLAlchemy documentation isn’t clear about what uses this tracking.

SQLALCHEMY_TRACK_MODIFICATIONS

If set to True (the default) Flask-SQLAlchemy will track modifications of objects and emit signals. This requires extra memory and can be disabled if not needed.

How do I find out if my project requires SQLALCHEMY_TRACK_MODIFICATIONS = True or if I can safely disable this feature and save memory on my server?


回答 0

您的应用程序很可能没有使用Flask-SQLAlchemy事件系统,因此可以安全地关闭它。您需要审核代码以进行验证-您正在寻找与models_committedbefore_models_committed挂钩的任何内容。如果确实发现您正在使用Flask-SQLAlchemy事件系统,则可能应该更新代码以使用SQLAlchemy的内置事件系统。

要关闭Flask-SQLAlchemy事件系统(并禁用警告),只需添加:

SQLALCHEMY_TRACK_MODIFICATIONS = False

更改为您的应用程序配置,直到更改默认设置为止(很有可能在Flask-SQLAlchemy v3中)。


背景-警告告诉您的是以下内容:

Flask-SQLAlchemy有自己的事件通知系统,该系统在SQLAlchemy之上分层。为此,它跟踪对SQLAlchemy会话的修改。这需要额外的资源,因此该选项SQLALCHEMY_TRACK_MODIFICATIONS允许您禁用修改跟踪系统。当前,该选项默认为True,但将来该默认值将更改为False,从而禁用事件系统。

据我了解,更改的理由有三点:

  1. 使用Flask-SQLAlchemy的事件系统的人并不多,但是大多数人没有意识到他们可以通过禁用它来节省系统资源。因此,更明智的默认设置是禁用它,想要它的人可以打开它。

  2. Flask-SQLAlchemy中的事件系统存在相当多的错误(请参阅下面提到的请求请求中与之相关的问题),需要为很少有人使用的功能进行额外的维护。

  3. 在v0.7中,SQLAlchemy本身添加了一个强大的事件系统,其中包括创建自定义事件的功能。理想情况下,Flask-SQLAlchemy事件系统除了创建一些自定义的SQLAlchemy事件挂钩和侦听器外,无所不用其事,然后让SQLAlchemy自己管理事件触发器。

您可以在有关拉动请求的讨论中看到更多信息,该请求开始触发此警告

Most likely your application doesn’t use the Flask-SQLAlchemy event system, so you’re probably safe to turn off. You’ll need to audit the code to verify–you’re looking for anything that hooks into models_committed or before_models_committed. If you do find that you’re using the Flask-SQLAlchemy event system, you probably should update the code to use SQLAlchemy’s built-in event system instead.

To turn off the Flask-SQLAlchemy event system (and disable the warning), just add:

SQLALCHEMY_TRACK_MODIFICATIONS = False

to your app config until the default is changed (most likely in Flask-SQLAlchemy v3).


Background–here’s what the warning is telling you:

Flask-SQLAlchemy has its own event notification system that gets layered on top of SQLAlchemy. To do this, it tracks modifications to the SQLAlchemy session. This takes extra resources, so the option SQLALCHEMY_TRACK_MODIFICATIONS allows you to disable the modification tracking system. Currently the option defaults to True, but in the future, that default will change to False, thereby disabling the event system.

As far as I understand, the rationale for the change is three-fold:

  1. Not many people use Flask-SQLAlchemy’s event system, but most people don’t realize they can save system resources by disabling it. So a saner default is to disable it and those who want it can turn it on.

  2. The event system in Flask-SQLAlchemy has been rather buggy (see issues linked to in the pull request mentioned below), requiring additional maintenance for a feature that few people use.

  3. In v0.7, SQLAlchemy itself added a powerful event system including the ability to create custom events. Ideally, the Flask-SQLAlchemy event system should do nothing more than create a few custom SQLAlchemy event hooks and listeners, and then let SQLAlchemy itself manage the event trigger.

You can see more in the discussion around the pull request that started triggering this warning.


回答 1

Jeff Widman的详细解释非常完美。

由于在完成此操作之前我曾进行过一些“复制粘贴”的操作,因此我想使下一个穿鞋的操作变得更容易。

在您的代码中,紧接在

app = Flask(__name__)

如果要启用轨道修改,只需添加:

app.config['SQLALCHEMY_TRACK_MODIFICATIONS'] = True

否则,如果您使用此功能,则可能需要将值更改为False,以免浪费系统资源。由于您仍在显式设置配置,因此这仍然会使警告保持沉默。

这是具有False值的相同代码段:

app.config['SQLALCHEMY_TRACK_MODIFICATIONS'] = False

感谢Jeff Widman提出的建议和详细信息。

Jeff Widman’s detailed explanation is simply perfect.

Since I had some copy’n’paste fights before getting this right I’d like to make it easier for the next one that will be in my shoes.

In your code, immediately after:

app = Flask(__name__)

If you want to enable track modifications simply add:

app.config['SQLALCHEMY_TRACK_MODIFICATIONS'] = True

Otherwise, if you are not using this feature, you may want to change the value to False in order not to waste system resources. This will still silence the warning since you’re anyway explicitly setting the config.

Here’s the same snippet with False value:

app.config['SQLALCHEMY_TRACK_MODIFICATIONS'] = False

Thanks to Jeff Widman for this added suggestion and details.


回答 2

上面的答案看起来不错。但是,我想在Flask-SQLAlchemy文档中指出这一行,因为SQLALCHEMY_TRACK_MODIFICATIONS = False在我的应用程序配置中设置后,我仍然收到这些警告。

在此页面上:http : //flask-sqlalchemy.pocoo.org/2.3/config/

Flask-SQLAlchemy存在以下配置值。Flask-SQLAlchemy从您的主要Flask配置中加载这些值,可以通过多种方式填充。请注意,其中一些不能在创建引擎后进行修改,因此请确保尽早进行配置,并且不要在运行时进行修改。

换句话说,app.config 创建Flask-SQLAlchemy数据库之前,请确保设置您的数据库。

例如,如果您将应用程序配置为set SQLALCHEMY_TRACK_MODIFICATIONS = False

from flask import Flask
app = Flask(__name__)
app.config['SQLALCHEMY_TRACK_MODIFICATIONS'] = False

db = SQLAlchemy(app)

The above answers look good. However, I wanted to point out this line in the Flask-SQLAlchemy documentation because I was still getting these warnings after setting SQLALCHEMY_TRACK_MODIFICATIONS = False in my application config.

On this page: http://flask-sqlalchemy.pocoo.org/2.3/config/

The following configuration values exist for Flask-SQLAlchemy. Flask-SQLAlchemy loads these values from your main Flask config which can be populated in various ways. Note that some of those cannot be modified after the engine was created so make sure to configure as early as possible and to not modify them at runtime.

In other words, make sure to set up your app.config before creating your Flask-SQLAlchemy database.

For example, if you are configuring your application to set SQLALCHEMY_TRACK_MODIFICATIONS = False:

from flask import Flask
app = Flask(__name__)
app.config['SQLALCHEMY_TRACK_MODIFICATIONS'] = False

db = SQLAlchemy(app)

如何更新SQLAlchemy行条目?

问题:如何更新SQLAlchemy行条目?

假设表有三列:usernamepasswordno_of_logins

当用户尝试登录时,将使用查询之类的内容检查条目

user = User.query.filter_by(username=form.username.data).first()

如果密码匹配,他将继续。我想做的是计算用户登录的次数。因此,无论他何时成功登录,我都希望增加该no_of_logins字段并将其存储回用户表。我不确定如何使用SqlAlchemy运行更新查询。

Assume table has three columns: username, password and no_of_logins.

When user tries to login, it’s checked for an entry with a query like

user = User.query.filter_by(username=form.username.data).first()

If password matches, he proceeds further. What I would like to do is count how many times the user logged in. Thus whenever he successfully logs in, I would like to increment the no_of_logins field and store it back to the user table. I’m not sure how to run update query with SqlAlchemy.


回答 0

user.no_of_logins += 1
session.commit()
user.no_of_logins += 1
session.commit()

回答 1

有几种UPDATE使用方法sqlalchemy

1) user.no_of_logins += 1
   session.commit()

2) session.query().\
       filter(User.username == form.username.data).\
       update({"no_of_logins": (User.no_of_logins +1)})
   session.commit()

3) conn = engine.connect()
   stmt = User.update().\
       values(no_of_logins=(User.no_of_logins + 1)).\
       where(User.username == form.username.data)
   conn.execute(stmt)

4) setattr(user, 'no_of_logins', user.no_of_logins+1)
   session.commit()

There are several ways to UPDATE using sqlalchemy

1) user.no_of_logins += 1
   session.commit()

2) session.query().\
       filter(User.username == form.username.data).\
       update({"no_of_logins": (User.no_of_logins +1)})
   session.commit()

3) conn = engine.connect()
   stmt = User.update().\
       values(no_of_logins=(User.no_of_logins + 1)).\
       where(User.username == form.username.data)
   conn.execute(stmt)

4) setattr(user, 'no_of_logins', user.no_of_logins+1)
   session.commit()

回答 2

举例说明澄清已接受答案的重要问题

直到我自己玩弄它之前,我都不了解它,所以我认为还会有其他人感到困惑。假设您正在id == 6no_of_logins == 30启动时的用户和用户一起工作。

# 1 (bad)
user.no_of_logins += 1
# result: UPDATE user SET no_of_logins = 31 WHERE user.id = 6

# 2 (bad)
user.no_of_logins = user.no_of_logins + 1
# result: UPDATE user SET no_of_logins = 31 WHERE user.id = 6

# 3 (bad)
setattr(user, 'no_of_logins', user.no_of_logins + 1)
# result: UPDATE user SET no_of_logins = 31 WHERE user.id = 6

# 4 (ok)
user.no_of_logins = User.no_of_logins + 1
# result: UPDATE user SET no_of_logins = no_of_logins + 1 WHERE user.id = 6

# 5 (ok)
setattr(user, 'no_of_logins', User.no_of_logins + 1)
# result: UPDATE user SET no_of_logins = no_of_logins + 1 WHERE user.id = 6

重点

通过引用类而不是实例,您可以使SQLAlchemy更加智能地进行增量,使其在数据库端而不是Python端发生。在数据库中执行此操作会更好,因为它不易受到数据损坏的影响(例如,两个客户端尝试同时进行增量操作,而最终结果仅是一次增量而不是两次增量)。我认为,如果您设置了锁或提高了隔离级别,则可以在Python中进行增量操作,但是如果不必这样做,为什么还要打扰呢?

注意事项

如果您要通过产生SQL之类的代码来增加两次SET no_of_logins = no_of_logins + 1,那么您将需要提交或至少刷新两次增加之间的增量,否则总共将只获得一个增量:

# 6 (bad)
user.no_of_logins = User.no_of_logins + 1
user.no_of_logins = User.no_of_logins + 1
session.commit()
# result: UPDATE user SET no_of_logins = no_of_logins + 1 WHERE user.id = 6

# 7 (ok)
user.no_of_logins = User.no_of_logins + 1
session.flush()
# result: UPDATE user SET no_of_logins = no_of_logins + 1 WHERE user.id = 6
user.no_of_logins = User.no_of_logins + 1
session.commit()
# result: UPDATE user SET no_of_logins = no_of_logins + 1 WHERE user.id = 6

Examples to clarify the important issue in accepted answer’s comments

I didn’t understand it until I played around with it myself, so I figured there would be others who were confused as well. Say you are working on the user whose id == 6 and whose no_of_logins == 30 when you start.

# 1 (bad)
user.no_of_logins += 1
# result: UPDATE user SET no_of_logins = 31 WHERE user.id = 6

# 2 (bad)
user.no_of_logins = user.no_of_logins + 1
# result: UPDATE user SET no_of_logins = 31 WHERE user.id = 6

# 3 (bad)
setattr(user, 'no_of_logins', user.no_of_logins + 1)
# result: UPDATE user SET no_of_logins = 31 WHERE user.id = 6

# 4 (ok)
user.no_of_logins = User.no_of_logins + 1
# result: UPDATE user SET no_of_logins = no_of_logins + 1 WHERE user.id = 6

# 5 (ok)
setattr(user, 'no_of_logins', User.no_of_logins + 1)
# result: UPDATE user SET no_of_logins = no_of_logins + 1 WHERE user.id = 6

The point

By referencing the class instead of the instance, you can get SQLAlchemy to be smarter about incrementing, getting it to happen on the database side instead of the Python side. Doing it within the database is better since it’s less vulnerable to data corruption (e.g. two clients attempt to increment at the same time with a net result of only one increment instead of two). I assume it’s possible to do the incrementing in Python if you set locks or bump up the isolation level, but why bother if you don’t have to?

A caveat

If you are going to increment twice via code that produces SQL like SET no_of_logins = no_of_logins + 1, then you will need to commit or at least flush in between increments, or else you will only get one increment in total:

# 6 (bad)
user.no_of_logins = User.no_of_logins + 1
user.no_of_logins = User.no_of_logins + 1
session.commit()
# result: UPDATE user SET no_of_logins = no_of_logins + 1 WHERE user.id = 6

# 7 (ok)
user.no_of_logins = User.no_of_logins + 1
session.flush()
# result: UPDATE user SET no_of_logins = no_of_logins + 1 WHERE user.id = 6
user.no_of_logins = User.no_of_logins + 1
session.commit()
# result: UPDATE user SET no_of_logins = no_of_logins + 1 WHERE user.id = 6

回答 3

借助user=User.query.filter_by(username=form.username.data).first()声明,您将获得指定用户的user变量。

现在,您可以像一样user.no_of_logins += 1更改新对象变量的值,并使用session的commit方法保存更改。

With the help of user=User.query.filter_by(username=form.username.data).first() statement you will get the specified user in user variable.

Now you can change the value of the new object variable like user.no_of_logins += 1 and save the changes with the session‘s commit method.


回答 4

我写了电报bot,并且在更新行时遇到了一些问题。如果您有模型,请使用此示例

def update_state(chat_id, state):
    try:
        value = Users.query.filter(Users.chat_id == str(chat_id)).first()
        value.state = str(state)
        db.session.flush()
        db.session.commit()
        #db.session.close()
    except:
        print('Error in def update_state')

为什么要使用db.session.flush()?这就是为什么>>> SQLAlchemy:flush()和commit()有什么区别?

I wrote telegram bot, and have some problem with update rows. Use this example, if you have Model

def update_state(chat_id, state):
    try:
        value = Users.query.filter(Users.chat_id == str(chat_id)).first()
        value.state = str(state)
        db.session.flush()
        db.session.commit()
        #db.session.close()
    except:
        print('Error in def update_state')

Why use db.session.flush()? That’s why >>> SQLAlchemy: What’s the difference between flush() and commit()?


SQLAlchemy:引擎,连接和会话的区别

问题:SQLAlchemy:引擎,连接和会话的区别

我使用SQLAlchemy并至少有三个实体:enginesession并且connection,其中有execute方法,所以如果我如想选择所有记录table我能做到这一点

engine.execute(select([table])).fetchall()

还有这个

connection.execute(select([table])).fetchall()

甚至这个

session.execute(select([table])).fetchall()

-结果将是相同的。

据我了解,如果有人使用engine.executeconnection,它会创建,打开session(Alchemy会为您处理)并执行查询。但是,执行此任务的这三种方式之间是否存在全局差异?

I use SQLAlchemy and there are at least three entities: engine, session and connection, which have execute method, so if I e.g. want to select all records from table I can do this

engine.execute(select([table])).fetchall()

and this

connection.execute(select([table])).fetchall()

and even this

session.execute(select([table])).fetchall()

– the results will be the same.

As I understand it, if someone uses engine.execute it creates connection, opens session (Alchemy takes care of it for you) and executes the query. But is there a global difference between these three ways of performing such a task?


回答 0

单行概述:

的行为execute()是在所有情况下相同,但它们是3种不同的方法,在EngineConnectionSession类。

到底是什么execute()

要了解行为,execute()我们需要调查Executable该类。Executable是所有“语句”类型对象的超类,包括select(),delete(),update(),insert(),text()-用最简单的词来说,Executable是SQLAlchemy支持的SQL表达式构造。

在所有情况下,该execute()方法均采用SQL文本或构造的SQL表达式,即SQLAlchemy支持的各种SQL表达式构造,并返回查询结果(ResultProxya-包装DB-API游标对象以更轻松地访问行列。)


为了进一步澄清(仅用于概念澄清,不建议使用方法)

除了Engine.execute()(无连接执行),Connection.execute()和之外Session.execute(),还可以execute()直接在任何Executable构造上使用。该Executable班有它自己的执行execute()-每个正式文件作为,对什么人一行说明execute()确实是“ 编译并执行这个Executable ”。在这种情况下,我们需要将Executable(SQL表达式构造)与Connection对象或Engine对象(隐式获取Connection对象)进行显式绑定,以便execute()将知道在何处执行SQL

下面的示例很好地演示了它-给定如下表:

from sqlalchemy import MetaData, Table, Column, Integer

meta = MetaData()
users_table = Table('users', meta,
    Column('id', Integer, primary_key=True),
    Column('name', String(50)))

显式执行,Connection.execute()-将SQL文本或构造的SQL表达式传递给以下execute()方法Connection

engine = create_engine('sqlite:///file.db')
connection = engine.connect()
result = connection.execute(users_table.select())
for row in result:
    # ....
connection.close()

显式无连接执行,Engine.execute()-将SQL文本或构造的SQL表达式直接传递给execute()Engine方法:

engine = create_engine('sqlite:///file.db')
result = engine.execute(users_table.select())
for row in result:
    # ....
result.close()

隐式执行(Executable.execute()-)也是无连接的,并且调用的execute()方法Executable,即它execute()直接在SQL表达式构造(的实例Executable)本身上调用方法。

engine = create_engine('sqlite:///file.db')
meta.bind = engine
result = users_table.select().execute()
for row in result:
    # ....
result.close()

注意:出于说明的目的,陈述了隐式执行示例-强烈建议不按照这种方式执行这种执行方式-按照docs

“隐式执行”是一个非常古老的使用模式,在大多数情况下,它比有用的方法更令人困惑,并且不鼓励使用它。两种模式似乎都鼓励在应用程序设计中过度使用权宜之计的“捷径”,这会在以后导致问题。


你的问题:

据我了解,如果有人使用engine.execute,它将创建连接,打开会话(Alchemy会为您关心)并执行查询。

您认为“如果有人使用engine.execute它会创建connection” 这一部分是正确的,但对于“打开session(炼金术会为您关心)并执行查询”而言,您是正确的- 在形式上,使用Engine.execute()Connection.execute()(几乎)是同一件事,在形式上,Connection对象是隐式创建的,在以后的情况下,我们显式实例化它。在这种情况下真正发生的是:

`Engine` object (instantiated via `create_engine()`) -> `Connection` object (instantiated via `engine_instance.connect()`) -> `connection.execute({*SQL expression*})`

但是,执行此任务的这三种方式之间是否存在全局差异?

在数据库层,这完全是同一回事,所有这些都在执行SQL(文本表达式或各种SQL表达式构造)。从应用程序的角度来看,有两个选项:

  • 直接执行-使用Engine.execute()Connection.execute()
  • 使用sessions-通过有效地处理交易单单元的工作,轻松session.add()session.rollback()session.commit()session.close()。在ORM(即映射表)的情况下,这是与DB进行交互的方式。提供identity_map,以便在单个请求期间立即获取已被访问的对象或新创建/添加的对象。

Session.execute()最终使用Connection.execute()语句执行方法来执行SQL语句。使用Session对象是SQLAlchemy ORM建议的应用程序与数据库交互的方式。

文档摘录:

重要的是要注意,在使用SQLAlchemy ORM时,通常不访问这些对象。而是将Session对象用作数据库的接口。但是,对于围绕直接使用文本SQL语句和/或SQL表达式构造而无需ORM更高级别的管理服务参与的应用程序,“引擎”和“连接”为王(也是王后?),请继续阅读。

A one-line overview:

The behavior of execute() is same in all the cases, but they are 3 different methods, in Engine, Connection, and Session classes.

What exactly is execute():

To understand behavior of execute() we need to look into the Executable class. Executable is a superclass for all “statement” types of objects, including select(), delete(),update(), insert(), text() – in simplest words possible, an Executable is a SQL expression construct supported in SQLAlchemy.

In all the cases the execute() method takes the SQL text or constructed SQL expression i.e. any of the variety of SQL expression constructs supported in SQLAlchemy and returns query results (a ResultProxy – Wraps a DB-API cursor object to provide easier access to row columns.)


To clarify it further (only for conceptual clarification, not a recommended approach):

In addition to Engine.execute() (connectionless execution), Connection.execute(), and Session.execute(), it is also possible to use the execute() directly on any Executable construct. The Executable class has it’s own implementation of execute() – As per official documentation, one line description about what the execute() does is “Compile and execute this Executable“. In this case we need to explicitly bind the Executable (SQL expression construct) with a Connection object or, Engine object (which implicitly get a Connection object), so the execute() will know where to execute the SQL.

The following example demonstrates it well – Given a table as below:

from sqlalchemy import MetaData, Table, Column, Integer

meta = MetaData()
users_table = Table('users', meta,
    Column('id', Integer, primary_key=True),
    Column('name', String(50)))

Explicit execution i.e. Connection.execute() – passing the SQL text or constructed SQL expression to the execute() method of Connection:

engine = create_engine('sqlite:///file.db')
connection = engine.connect()
result = connection.execute(users_table.select())
for row in result:
    # ....
connection.close()

Explicit connectionless execution i.e. Engine.execute() – passing the SQL text or constructed SQL expression directly to the execute() method of Engine:

engine = create_engine('sqlite:///file.db')
result = engine.execute(users_table.select())
for row in result:
    # ....
result.close()

Implicit execution i.e. Executable.execute() – is also connectionless, and calls the execute() method of the Executable, that is, it calls execute() method directly on the SQL expression construct (an instance of Executable) itself.

engine = create_engine('sqlite:///file.db')
meta.bind = engine
result = users_table.select().execute()
for row in result:
    # ....
result.close()

Note: Stated the implicit execution example for the purpose of clarification – this way of execution is highly not recommended – as per docs:

“implicit execution” is a very old usage pattern that in most cases is more confusing than it is helpful, and its usage is discouraged. Both patterns seem to encourage the overuse of expedient “short cuts” in application design which lead to problems later on.


Your questions:

As I understand if someone use engine.execute it creates connection, opens session (Alchemy cares about it for you) and executes query.

You’re right for the part “if someone use engine.execute it creates connection ” but not for “opens session (Alchemy cares about it for you) and executes query ” – Using Engine.execute() and Connection.execute() is (almost) one the same thing, in formal, Connection object gets created implicitly, and in later case we explicitly instantiate it. What really happens in this case is:

`Engine` object (instantiated via `create_engine()`) -> `Connection` object (instantiated via `engine_instance.connect()`) -> `connection.execute({*SQL expression*})`

But is there a global difference between these three ways of performing such task?

At DB layer it’s exactly the same thing, all of them are executing SQL (text expression or various SQL expression constructs). From application’s point of view there are two options:

  • Direct execution – Using Engine.execute() or Connection.execute()
  • Using sessions – efficiently handles transaction as single unit-of-work, with ease via session.add(), session.rollback(), session.commit(), session.close(). It is the way to interact with the DB in case of ORM i.e. mapped tables. Provides identity_map for instantly getting already accessed or newly created/added objects during a single request.

Session.execute() ultimately uses Connection.execute() statement execution method in order to execute the SQL statement. Using Session object is SQLAlchemy ORM’s recommended way for an application to interact with the database.

An excerpt from the docs:

Its important to note that when using the SQLAlchemy ORM, these objects are not generally accessed; instead, the Session object is used as the interface to the database. However, for applications that are built around direct usage of textual SQL statements and/or SQL expression constructs without involvement by the ORM’s higher level management services, the Engine and Connection are king (and queen?) – read on.


回答 1

Nabeel的答案涵盖了很多细节并且很有帮助,但是我发现难以理解。由于这是该问题的第一个Google结果,因此,我对以后发现此问题的人们加深了理解:

运行.execute()

正如OP和Nabell Ahmed都指出的那样,执行平原时SELECT * FROM tablename,提供的结果没有区别。

这三个对象之间的区别取决于上下文就成为非常重要的SELECT声明中,或者更常见的是,当你想要做其他事情一样使用INSERTDELETE等等。

何时使用引擎,连接,会话

  • 引擎是SQLAlchemy使用的最低级别的对象。它维护了一个连接池,可在应用程序需要与数据库对话时使用。.execute()是一种先调用conn = engine.connect(close_with_result=True)然后调用的便捷方法conn.execute()。close_with_result参数表示连接自动关闭。(我稍微解释了源代码,但本质上是正确的)。编辑:这是engine.execute的源代码

    您可以使用引擎执行原始SQL。

    result = engine.execute('SELECT * FROM tablename;')
    #what engine.execute() is doing under the hood
    conn = engine.connect(close_with_result=True)
    result = conn.execute('SELECT * FROM tablename;')
    
    #after you iterate over the results, the result and connection get closed
    for row in result:
        print(result['columnname']
    
    #or you can explicitly close the result, which also closes the connection
    result.close()

    基本用法下的文档中对此进行了介绍。

  • 连接(正如我们在上面看到的)实际上是执行SQL查询的工作。每当您想更好地控制连接的属性,何时关闭连接等时,都应该执行此操作。例如,非常重要的示例是Transaction,它使您可以决定何时将更改提交到数据库。在正常使用中,更改是自动提交的。通过使用事务,您可以(例如)运行多个不同的SQL语句,如果其中一个出现问题,则可以立即撤消所有更改。

    connection = engine.connect()
    trans = connection.begin()
    try:
        connection.execute("INSERT INTO films VALUES ('Comedy', '82 minutes');")
        connection.execute("INSERT INTO datalog VALUES ('added a comedy');")
        trans.commit()
    except:
        trans.rollback()
        raise

    如果一次失败,这将使您撤消两项更改,就像您忘记创建数据日志表一样。

    因此,如果您正在执行原始SQL代码并需要控制,请使用连接

  • 会话用于SQLAlchemy的对象关系管理(ORM)方面(实际上,您可以从它们的导入方式中看到这一点:)from sqlalchemy.orm import sessionmaker。他们在后台使用连接和事务来运行其自动生成的SQL语句。.execute()是一个便捷功能,可传递到会话绑定的任何对象(通常是引擎,但可以是连接)。

    如果您使用的是ORM功能,请使用会话。如果只执行不绑定对象的直接SQL查询,则最好直接使用连接。

Nabeel’s answer covers a lot of details and is helpful, but I found it confusing to follow. Since this is currently the first Google result for this issue, adding my understanding of it for future people that find this question:

Running .execute()

As OP and Nabell Ahmed both note, when executing a plain SELECT * FROM tablename, there’s no difference in the result provided.

The differences between these three objects do become important depending on the context that the SELECT statement is used in or, more commonly, when you want to do other things like INSERT, DELETE, etc.

When to use Engine, Connection, Session generally

  • Engine is the lowest level object used by SQLAlchemy. It maintains a pool of connections available for use whenever the application needs to talk to the database. .execute() is a convenience method that first calls conn = engine.connect(close_with_result=True) and the then conn.execute(). The close_with_result parameter means the connection is closed automatically. (I’m slightly paraphrasing the source code, but essentially true). edit: Here’s the source code for engine.execute

    You can use engine to execute raw SQL.

    result = engine.execute('SELECT * FROM tablename;')
    #what engine.execute() is doing under the hood
    conn = engine.connect(close_with_result=True)
    result = conn.execute('SELECT * FROM tablename;')
    
    #after you iterate over the results, the result and connection get closed
    for row in result:
        print(result['columnname']
    
    #or you can explicitly close the result, which also closes the connection
    result.close()
    

    This is covered in the docs under basic usage.

  • Connection is (as we saw above) the thing that actually does the work of executing a SQL query. You should do this whenever you want greater control over attributes of the connection, when it gets closed, etc. For example, a very import example of this is a Transaction, which lets you decide when to commit your changes to the database. In normal use, changes are autocommitted. With the use of transactions, you could (for example) run several different SQL statements and if something goes wrong with one of them you could undo all the changes at once.

    connection = engine.connect()
    trans = connection.begin()
    try:
        connection.execute("INSERT INTO films VALUES ('Comedy', '82 minutes');")
        connection.execute("INSERT INTO datalog VALUES ('added a comedy');")
        trans.commit()
    except:
        trans.rollback()
        raise
    

    This would let you undo both changes if one failed, like if you forgot to create the datalog table.

    So if you’re executing raw SQL code and need control, use connections

  • Sessions are used for the Object Relationship Management (ORM) aspect of SQLAlchemy (in fact you can see this from how they’re imported: from sqlalchemy.orm import sessionmaker). They use connections and transactions under the hood to run their automatically-generated SQL statements. .execute() is a convenience function that passes through to whatever the session is bound to (usually an engine, but can be a connection).

    If you’re using the ORM functionality, use session; if you’re only doing straight SQL queries not bound to objects, you’re probably better off using connections directly.


回答 2

这是运行诸如GRANT之类的DCL(数据控制语言)的示例

def grantAccess(db, tb, user):
  import sqlalchemy as SA
  import psycopg2

  url = "{d}+{driver}://{u}:{p}@{h}:{port}/{db}".\
            format(d="redshift",
            driver='psycopg2',
            u=username,
            p=password,
            h=host,
            port=port,
            db=db)
  engine = SA.create_engine(url)
  cnn = engine.connect()
  trans = cnn.begin()
  strSQL = "GRANT SELECT on table " + tb + " to " + user + " ;"
  try:
      cnn.execute(strSQL)
      trans.commit()
  except:
      trans.rollback()
      raise

Here is an example of running DCL (Data Control Language) such as GRANT

def grantAccess(db, tb, user):
  import sqlalchemy as SA
  import psycopg2

  url = "{d}+{driver}://{u}:{p}@{h}:{port}/{db}".\
            format(d="redshift",
            driver='psycopg2',
            u=username,
            p=password,
            h=host,
            port=port,
            db=db)
  engine = SA.create_engine(url)
  cnn = engine.connect()
  trans = cnn.begin()
  strSQL = "GRANT SELECT on table " + tb + " to " + user + " ;"
  try:
      cnn.execute(strSQL)
      trans.commit()
  except:
      trans.rollback()
      raise

使用SQLAlchemy ORM批量插入

问题:使用SQLAlchemy ORM批量插入

有什么方法可以让SQLAlchemy进行批量插入,而不是插入每个对象。即

在做:

INSERT INTO `foo` (`bar`) VALUES (1), (2), (3)

而不是:

INSERT INTO `foo` (`bar`) VALUES (1)
INSERT INTO `foo` (`bar`) VALUES (2)
INSERT INTO `foo` (`bar`) VALUES (3)

我刚刚将一些代码转换为使用sqlalchemy而不是原始sql,尽管现在使用它起来要好得多,但现在似乎要慢一些(最多10倍),我想知道这是否是原因。

也许我可以更有效地使用会话来改善这种情况。目前,我已经添加了一些东西,autoCommit=False并做了一个session.commit()。尽管如果在其他地方更改了数据库,这似乎会使数据过时,例如,即使我执行新查询,我仍然可以返回旧结果?

谢谢你的帮助!

Is there any way to get SQLAlchemy to do a bulk insert rather than inserting each individual object. i.e.,

doing:

INSERT INTO `foo` (`bar`) VALUES (1), (2), (3)

rather than:

INSERT INTO `foo` (`bar`) VALUES (1)
INSERT INTO `foo` (`bar`) VALUES (2)
INSERT INTO `foo` (`bar`) VALUES (3)

I’ve just converted some code to use sqlalchemy rather than raw sql and although it is now much nicer to work with it seems to be slower now (up to a factor of 10), I’m wondering if this is the reason.

May be I could improve the situation using sessions more efficiently. At the moment I have autoCommit=False and do a session.commit() after I’ve added some stuff. Although this seems to cause the data to go stale if the DB is changed elsewhere, like even if I do a new query I still get old results back?

Thanks for your help!


回答 0

SQLAlchemy在版本中引入了该功能1.0.0

批量操作-SQLAlchemy文档

通过这些操作,您现在可以批量插入或更新!

例如,您可以执行以下操作:

s = Session()
objects = [
    User(name="u1"),
    User(name="u2"),
    User(name="u3")
]
s.bulk_save_objects(objects)
s.commit()

在这里,将制成大量插入物。

SQLAlchemy introduced that in version 1.0.0:

Bulk operations – SQLAlchemy docs

With these operations, you can now do bulk inserts or updates!

For instance, you can do:

s = Session()
objects = [
    User(name="u1"),
    User(name="u2"),
    User(name="u3")
]
s.bulk_save_objects(objects)
s.commit()

Here, a bulk insert will be made.


回答 1

sqlalchemy文档对可用于批量插入的各种技术的性能进行了总结

ORM基本上不是用于高性能批量插入的-这是SQLAlchemy除了将ORM作为一流组件之外还提供Core的全部原因。

对于快速批量插入的用例,ORM所基于的SQL生成和执行系统是Core的一部分。直接使用该系统,我们可以产生与直接使用原始数据库API相比具有竞争力的INSERT。

另外,SQLAlchemy ORM提供了Bulk Operations方法套件,该套件提供了到工作单元过程各部分的挂钩,以便发出基于ORM的自动化程度较低的Core级INSERT和UPDATE构造。

下面的示例说明了基于时间的测试,该测试针对从自动程度最高到最少的几种不同的行插入方法。使用cPython 2.7,可以观察到运行时:

classics-MacBook-Pro:sqlalchemy classic$ python test.py
SQLAlchemy ORM: Total time for 100000 records 12.0471920967 secs
SQLAlchemy ORM pk given: Total time for 100000 records 7.06283402443 secs
SQLAlchemy ORM bulk_save_objects(): Total time for 100000 records 0.856323003769 secs
SQLAlchemy Core: Total time for 100000 records 0.485800027847 secs
sqlite3: Total time for 100000 records 0.487842082977 sec

脚本:

import time
import sqlite3

from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import Column, Integer, String,  create_engine
from sqlalchemy.orm import scoped_session, sessionmaker

Base = declarative_base()
DBSession = scoped_session(sessionmaker())
engine = None


class Customer(Base):
    __tablename__ = "customer"
    id = Column(Integer, primary_key=True)
    name = Column(String(255))


def init_sqlalchemy(dbname='sqlite:///sqlalchemy.db'):
    global engine
    engine = create_engine(dbname, echo=False)
    DBSession.remove()
    DBSession.configure(bind=engine, autoflush=False, expire_on_commit=False)
    Base.metadata.drop_all(engine)
    Base.metadata.create_all(engine)


def test_sqlalchemy_orm(n=100000):
    init_sqlalchemy()
    t0 = time.time()
    for i in xrange(n):
        customer = Customer()
        customer.name = 'NAME ' + str(i)
        DBSession.add(customer)
        if i % 1000 == 0:
            DBSession.flush()
    DBSession.commit()
    print(
        "SQLAlchemy ORM: Total time for " + str(n) +
        " records " + str(time.time() - t0) + " secs")


def test_sqlalchemy_orm_pk_given(n=100000):
    init_sqlalchemy()
    t0 = time.time()
    for i in xrange(n):
        customer = Customer(id=i+1, name="NAME " + str(i))
        DBSession.add(customer)
        if i % 1000 == 0:
            DBSession.flush()
    DBSession.commit()
    print(
        "SQLAlchemy ORM pk given: Total time for " + str(n) +
        " records " + str(time.time() - t0) + " secs")


def test_sqlalchemy_orm_bulk_insert(n=100000):
    init_sqlalchemy()
    t0 = time.time()
    n1 = n
    while n1 > 0:
        n1 = n1 - 10000
        DBSession.bulk_insert_mappings(
            Customer,
            [
                dict(name="NAME " + str(i))
                for i in xrange(min(10000, n1))
            ]
        )
    DBSession.commit()
    print(
        "SQLAlchemy ORM bulk_save_objects(): Total time for " + str(n) +
        " records " + str(time.time() - t0) + " secs")


def test_sqlalchemy_core(n=100000):
    init_sqlalchemy()
    t0 = time.time()
    engine.execute(
        Customer.__table__.insert(),
        [{"name": 'NAME ' + str(i)} for i in xrange(n)]
    )
    print(
        "SQLAlchemy Core: Total time for " + str(n) +
        " records " + str(time.time() - t0) + " secs")


def init_sqlite3(dbname):
    conn = sqlite3.connect(dbname)
    c = conn.cursor()
    c.execute("DROP TABLE IF EXISTS customer")
    c.execute(
        "CREATE TABLE customer (id INTEGER NOT NULL, "
        "name VARCHAR(255), PRIMARY KEY(id))")
    conn.commit()
    return conn


def test_sqlite3(n=100000, dbname='sqlite3.db'):
    conn = init_sqlite3(dbname)
    c = conn.cursor()
    t0 = time.time()
    for i in xrange(n):
        row = ('NAME ' + str(i),)
        c.execute("INSERT INTO customer (name) VALUES (?)", row)
    conn.commit()
    print(
        "sqlite3: Total time for " + str(n) +
        " records " + str(time.time() - t0) + " sec")

if __name__ == '__main__':
    test_sqlalchemy_orm(100000)
    test_sqlalchemy_orm_pk_given(100000)
    test_sqlalchemy_orm_bulk_insert(100000)
    test_sqlalchemy_core(100000)
    test_sqlite3(100000)

The sqlalchemy docs have a writeup on the performance of various techniques that can be used for bulk inserts:

ORMs are basically not intended for high-performance bulk inserts – this is the whole reason SQLAlchemy offers the Core in addition to the ORM as a first-class component.

For the use case of fast bulk inserts, the SQL generation and execution system that the ORM builds on top of is part of the Core. Using this system directly, we can produce an INSERT that is competitive with using the raw database API directly.

Alternatively, the SQLAlchemy ORM offers the Bulk Operations suite of methods, which provide hooks into subsections of the unit of work process in order to emit Core-level INSERT and UPDATE constructs with a small degree of ORM-based automation.

The example below illustrates time-based tests for several different methods of inserting rows, going from the most automated to the least. With cPython 2.7, runtimes observed:

classics-MacBook-Pro:sqlalchemy classic$ python test.py
SQLAlchemy ORM: Total time for 100000 records 12.0471920967 secs
SQLAlchemy ORM pk given: Total time for 100000 records 7.06283402443 secs
SQLAlchemy ORM bulk_save_objects(): Total time for 100000 records 0.856323003769 secs
SQLAlchemy Core: Total time for 100000 records 0.485800027847 secs
sqlite3: Total time for 100000 records 0.487842082977 sec

Script:

import time
import sqlite3

from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import Column, Integer, String,  create_engine
from sqlalchemy.orm import scoped_session, sessionmaker

Base = declarative_base()
DBSession = scoped_session(sessionmaker())
engine = None


class Customer(Base):
    __tablename__ = "customer"
    id = Column(Integer, primary_key=True)
    name = Column(String(255))


def init_sqlalchemy(dbname='sqlite:///sqlalchemy.db'):
    global engine
    engine = create_engine(dbname, echo=False)
    DBSession.remove()
    DBSession.configure(bind=engine, autoflush=False, expire_on_commit=False)
    Base.metadata.drop_all(engine)
    Base.metadata.create_all(engine)


def test_sqlalchemy_orm(n=100000):
    init_sqlalchemy()
    t0 = time.time()
    for i in xrange(n):
        customer = Customer()
        customer.name = 'NAME ' + str(i)
        DBSession.add(customer)
        if i % 1000 == 0:
            DBSession.flush()
    DBSession.commit()
    print(
        "SQLAlchemy ORM: Total time for " + str(n) +
        " records " + str(time.time() - t0) + " secs")


def test_sqlalchemy_orm_pk_given(n=100000):
    init_sqlalchemy()
    t0 = time.time()
    for i in xrange(n):
        customer = Customer(id=i+1, name="NAME " + str(i))
        DBSession.add(customer)
        if i % 1000 == 0:
            DBSession.flush()
    DBSession.commit()
    print(
        "SQLAlchemy ORM pk given: Total time for " + str(n) +
        " records " + str(time.time() - t0) + " secs")


def test_sqlalchemy_orm_bulk_insert(n=100000):
    init_sqlalchemy()
    t0 = time.time()
    n1 = n
    while n1 > 0:
        n1 = n1 - 10000
        DBSession.bulk_insert_mappings(
            Customer,
            [
                dict(name="NAME " + str(i))
                for i in xrange(min(10000, n1))
            ]
        )
    DBSession.commit()
    print(
        "SQLAlchemy ORM bulk_save_objects(): Total time for " + str(n) +
        " records " + str(time.time() - t0) + " secs")


def test_sqlalchemy_core(n=100000):
    init_sqlalchemy()
    t0 = time.time()
    engine.execute(
        Customer.__table__.insert(),
        [{"name": 'NAME ' + str(i)} for i in xrange(n)]
    )
    print(
        "SQLAlchemy Core: Total time for " + str(n) +
        " records " + str(time.time() - t0) + " secs")


def init_sqlite3(dbname):
    conn = sqlite3.connect(dbname)
    c = conn.cursor()
    c.execute("DROP TABLE IF EXISTS customer")
    c.execute(
        "CREATE TABLE customer (id INTEGER NOT NULL, "
        "name VARCHAR(255), PRIMARY KEY(id))")
    conn.commit()
    return conn


def test_sqlite3(n=100000, dbname='sqlite3.db'):
    conn = init_sqlite3(dbname)
    c = conn.cursor()
    t0 = time.time()
    for i in xrange(n):
        row = ('NAME ' + str(i),)
        c.execute("INSERT INTO customer (name) VALUES (?)", row)
    conn.commit()
    print(
        "sqlite3: Total time for " + str(n) +
        " records " + str(time.time() - t0) + " sec")

if __name__ == '__main__':
    test_sqlalchemy_orm(100000)
    test_sqlalchemy_orm_pk_given(100000)
    test_sqlalchemy_orm_bulk_insert(100000)
    test_sqlalchemy_core(100000)
    test_sqlite3(100000)

回答 2

据我所知,没有办法让ORM发出批量插入。我认为根本原因是SQLAlchemy需要跟踪每个对象的身份(即新的主键),而大容量插入会对此产生干扰。例如,假设您的foo表包含一id列并映射到一个Foo类:

x = Foo(bar=1)
print x.id
# None
session.add(x)
session.flush()
# BEGIN
# INSERT INTO foo (bar) VALUES(1)
# COMMIT
print x.id
# 1

由于SQLAlchemy在x.id不发出另一个查询的情况下获取了该值,因此我们可以推断出它直接从该INSERT语句中获取了该值。如果不需要随后通过相同实例访问创建的对象,则可以跳过ORM层进行插入:

Foo.__table__.insert().execute([{'bar': 1}, {'bar': 2}, {'bar': 3}])
# INSERT INTO foo (bar) VALUES ((1,), (2,), (3,))

SQLAlchemy无法将这些新行与任何现有对象匹配,因此您必须重新查询它们以进行任何后续操作。

至于过时的数据,记住该会话没有内置的方式来了解何时在会话外更改数据库是很有帮助的。为了通过现有实例访问外部修改的数据,必须将这些实例标记为expired。默认情况下会发生这种情况session.commit(),但可以通过调用session.expire_all()或手动完成session.expire(instance)。一个例子(省略SQL):

x = Foo(bar=1)
session.add(x)
session.commit()
print x.bar
# 1
foo.update().execute(bar=42)
print x.bar
# 1
session.expire(x)
print x.bar
# 42

session.commit()expires x,因此第一个打印语句隐式打开一个新事务并重新查询x属性。如果注释掉第一个打印语句,您会注意到第二个打印语句现在会选择正确的值,因为直到更新后才会发出新查询。

从事务隔离的角度来看,这是有道理的-您只应在事务之间进行外部修改。如果这给您带来麻烦,建议您弄清或重新考虑应用程序的事务边界,而不要立即进行操作session.expire_all()

As far as I know, there is no way to get the ORM to issue bulk inserts. I believe the underlying reason is that SQLAlchemy needs to keep track of each object’s identity (i.e., new primary keys), and bulk inserts interfere with that. For example, assuming your foo table contains an id column and is mapped to a Foo class:

x = Foo(bar=1)
print x.id
# None
session.add(x)
session.flush()
# BEGIN
# INSERT INTO foo (bar) VALUES(1)
# COMMIT
print x.id
# 1

Since SQLAlchemy picked up the value for x.id without issuing another query, we can infer that it got the value directly from the INSERT statement. If you don’t need subsequent access to the created objects via the same instances, you can skip the ORM layer for your insert:

Foo.__table__.insert().execute([{'bar': 1}, {'bar': 2}, {'bar': 3}])
# INSERT INTO foo (bar) VALUES ((1,), (2,), (3,))

SQLAlchemy can’t match these new rows with any existing objects, so you’ll have to query them anew for any subsequent operations.

As far as stale data is concerned, it’s helpful to remember that the session has no built-in way to know when the database is changed outside of the session. In order to access externally modified data through existing instances, the instances must be marked as expired. This happens by default on session.commit(), but can be done manually by calling session.expire_all() or session.expire(instance). An example (SQL omitted):

x = Foo(bar=1)
session.add(x)
session.commit()
print x.bar
# 1
foo.update().execute(bar=42)
print x.bar
# 1
session.expire(x)
print x.bar
# 42

session.commit() expires x, so the first print statement implicitly opens a new transaction and re-queries x‘s attributes. If you comment out the first print statement, you’ll notice that the second one now picks up the correct value, because the new query isn’t emitted until after the update.

This makes sense from the point of view of transactional isolation – you should only pick up external modifications between transactions. If this is causing you trouble, I’d suggest clarifying or re-thinking your application’s transaction boundaries instead of immediately reaching for session.expire_all().


回答 3

我通常使用add_all

from app import session
from models import User

objects = [User(name="u1"), User(name="u2"), User(name="u3")]
session.add_all(objects)
session.commit()

I usually do it using add_all.

from app import session
from models import User

objects = [User(name="u1"), User(name="u2"), User(name="u3")]
session.add_all(objects)
session.commit()

回答 4

从0.8版开始,直接支持已添加到SQLAlchemy

根据docsconnection.execute(table.insert().values(data))应该可以解决问题。(请注意,这是一样的connection.execute(table.insert(), data)通过将呼叫这导致许多个别行插入executemany)。除了本地连接之外,其他任何方面的性能差异都可能很大。

Direct support was added to SQLAlchemy as of version 0.8

As per the docs, connection.execute(table.insert().values(data)) should do the trick. (Note that this is not the same as connection.execute(table.insert(), data) which results in many individual row inserts via a call to executemany). On anything but a local connection the difference in performance can be enormous.


回答 5

SQLAlchemy在版本中引入了该功能1.0.0

批量操作-SQLAlchemy文档

通过这些操作,您现在可以批量插入或更新!

例如(如果您希望简单表INSERT的开销最小),可以使用Session.bulk_insert_mappings()

loadme = [(1, 'a'),
          (2, 'b'),
          (3, 'c')]
dicts = [dict(bar=t[0], fly=t[1]) for t in loadme]

s = Session()
s.bulk_insert_mappings(Foo, dicts)
s.commit()

或者,如果需要,可以跳过loadme元组,直接将字典写进去dicts(但是我发现将所有的单词遗漏在数据之外并循环加载字典列表会更容易)。

SQLAlchemy introduced that in version 1.0.0:

Bulk operations – SQLAlchemy docs

With these operations, you can now do bulk inserts or updates!

For instance (if you want the lowest overhead for simple table INSERTs), you can use Session.bulk_insert_mappings():

loadme = [(1, 'a'),
          (2, 'b'),
          (3, 'c')]
dicts = [dict(bar=t[0], fly=t[1]) for t in loadme]

s = Session()
s.bulk_insert_mappings(Foo, dicts)
s.commit()

Or, if you want, skip the loadme tuples and write the dictionaries directly into dicts (but I find it easier to leave all the wordiness out of the data and load up a list of dictionaries in a loop).


回答 6

Piere的回答是正确的,但是一个问题是bulk_save_objects,如果您担心的话,默认情况下不会返回对象的主键。设置return_defaultsTrue可得到此行为。

文档在这里

foos = [Foo(bar='a',), Foo(bar='b'), Foo(bar='c')]
session.bulk_save_objects(foos, return_defaults=True)
for foo in foos:
    assert foo.id is not None
session.commit()

Piere’s answer is correct but one issue is that bulk_save_objects by default does not return the primary keys of the objects, if that is of concern to you. Set return_defaults to True to get this behavior.

The documentation is here.

foos = [Foo(bar='a',), Foo(bar='b'), Foo(bar='c')]
session.bulk_save_objects(foos, return_defaults=True)
for foo in foos:
    assert foo.id is not None
session.commit()

回答 7

条条大路通罗马,但其中一些横穿山脉,需要渡轮,但是如果您想快速到达那儿,只需上高速公路。


在这种情况下,高速公路将使用psycopg2execute_batch()功能。该文档说的最好:

当前的实现executemany()(使用非常慈善的轻描淡写)不是特别有效。这些功能可用于加快针对一组参数的语句的重复执行。通过减少服务器往返次数,性能可以比使用服务器好几个数量级。executemany()

在我自己的测试execute_batch()快2倍左右executemany(),并给出配置进行进一步的调整所以page_size的选项(如果你想挤进业绩的最后2-3%的驾驶者)。

如果使用SQLAlchemy,则可以通过use_batch_mode=True在实例化引擎时将其设置为参数来轻松启用相同功能。create_engine()

All Roads Lead to Rome, but some of them crosses mountains, requires ferries but if you want to get there quickly just take the motorway.


In this case the motorway is to use the execute_batch() feature of psycopg2. The documentation says it the best:

The current implementation of executemany() is (using an extremely charitable understatement) not particularly performing. These functions can be used to speed up the repeated execution of a statement against a set of parameters. By reducing the number of server roundtrips the performance can be orders of magnitude better than using executemany().

In my own test execute_batch() is approximately twice as fast as executemany(), and gives the option to configure the page_size for further tweaking (if you want to squeeze the last 2-3% of performance out of the driver).

The same feature can easily be enabled if you are using SQLAlchemy by setting use_batch_mode=True as a parameter when you instantiate the engine with create_engine()


回答 8

这是一种方法:

values = [1, 2, 3]
Foo.__table__.insert().execute([{'bar': x} for x in values])

这样插入:

INSERT INTO `foo` (`bar`) VALUES (1), (2), (3)

参考:SQLAlchemy FAQ包含各种提交方法的基准。

This is a way:

values = [1, 2, 3]
Foo.__table__.insert().execute([{'bar': x} for x in values])

This will insert like this:

INSERT INTO `foo` (`bar`) VALUES (1), (2), (3)

Reference: The SQLAlchemy FAQ includes benchmarks for various commit methods.


回答 9

到目前为止,我发现的最佳答案是在sqlalchemy文档中:

http://docs.sqlalchemy.org/en/latest/faq/performance.html#im-inserting-400-000-rows-with-the-orm-and-it-s-really-slow

有一个完整示例说明了可能的解决方案基准。

如文档所示:

bulk_save_objects不是最佳解决方案,但其性能是正确的。

就可读性而言,第二好的实现是我认为使用SQLAlchemy Core:

def test_sqlalchemy_core(n=100000):
    init_sqlalchemy()
    t0 = time.time()
    engine.execute(
        Customer.__table__.insert(),
            [{"name": 'NAME ' + str(i)} for i in xrange(n)]
    )

文档文章中提供了此功能的上下文。

The best answer I found so far was in sqlalchemy documentation:

http://docs.sqlalchemy.org/en/latest/faq/performance.html#i-m-inserting-400-000-rows-with-the-orm-and-it-s-really-slow

There is a complete example of a benchmark of possible solutions.

As shown in the documentation:

bulk_save_objects is not the best solution but it performance are correct.

The second best implementation in terms of readability I think was with the SQLAlchemy Core:

def test_sqlalchemy_core(n=100000):
    init_sqlalchemy()
    t0 = time.time()
    engine.execute(
        Customer.__table__.insert(),
            [{"name": 'NAME ' + str(i)} for i in xrange(n)]
    )

The context of this function is given in the documentation article.


如何在Flask-SQLAlchemy中按ID删除记录

问题:如何在Flask-SQLAlchemy中按ID删除记录

users在MySql数据库中有表。这个表有idname而且age领域。

如何删除某些记录id

现在,我使用以下代码:

user = User.query.get(id)
db.session.delete(user)
db.session.commit()

但是我不想在删除操作之前进行任何查询。有什么办法吗?我知道,我可以使用db.engine.execute("delete from users where id=..."),但是我想使用delete()方法。

I have users table in my MySql database. This table has id, name and age fields.

How can I delete some record by id?

Now I use the following code:

user = User.query.get(id)
db.session.delete(user)
db.session.commit()

But I don’t want to make any query before delete operation. Is there any way to do this? I know, I can use db.engine.execute("delete from users where id=..."), but I would like to use delete() method.


回答 0

你可以这样做,

User.query.filter_by(id=123).delete()

要么

User.query.filter(User.id == 123).delete()

确保作出commitdelete()生效。

You can do this,

User.query.filter_by(id=123).delete()

or

User.query.filter(User.id == 123).delete()

Make sure to commit for delete() to take effect.


回答 1

只想分享另一个选择:

# mark two objects to be deleted
session.delete(obj1)
session.delete(obj2)

# commit (or flush)
session.commit()

http://docs.sqlalchemy.org/en/latest/orm/session_basics.html#deleting

在此示例中,以下代码可以正常工作:

obj = User.query.filter_by(id=123).one()
session.delete(obj)
session.commit()

Just want to share another option:

# mark two objects to be deleted
session.delete(obj1)
session.delete(obj2)

# commit (or flush)
session.commit()

http://docs.sqlalchemy.org/en/latest/orm/session_basics.html#deleting

In this example, the following codes shall works fine:

obj = User.query.filter_by(id=123).one()
session.delete(obj)
session.commit()

回答 2

另一个可能的解决方案,特别是如果要批量删除

deleted_objects = User.__table__.delete().where(User.id.in_([1, 2, 3]))
session.execute(deleted_objects)
session.commit()

Another possible solution specially if you want batch delete

deleted_objects = User.__table__.delete().where(User.id.in_([1, 2, 3]))
session.execute(deleted_objects)
session.commit()

Flask SQLAlchemy查询,指定列名

问题:Flask SQLAlchemy查询,指定列名

如何使用模型在查询中指定所需的列(默认情况下会选择所有列)?我知道如何使用sqlalchmey会话:session.query(self.col1),但是如何使用模型呢?我做不到SomeModel.query()。有办法吗?

How do I specify the column that I want in my query using a model (it selects all columns by default)? I know how to do this with the sqlalchmey session: session.query(self.col1), but how do I do it with with models? I can’t do SomeModel.query(). Is there a way?


回答 0

您可以使用该with_entities()方法来限制要返回结果的列。(文件

result = SomeModel.query.with_entities(SomeModel.col1, SomeModel.col2)

根据您的要求,您可能还会发现递延有用。它们使您可以返回完整的对象,但可以限制导线上的列。

You can use the with_entities() method to restrict which columns you’d like to return in the result. (documentation)

result = SomeModel.query.with_entities(SomeModel.col1, SomeModel.col2)

Depending on your requirements, you may also find deferreds useful. They allow you to return the full object but restrict the columns that come over the wire.


回答 1

session.query().with_entities(SomeModel.col1)

是相同的

session.query(SomeModel.col1)

对于别名,我们可以使用.label()

session.query(SomeModel.col1.label('some alias name'))
session.query().with_entities(SomeModel.col1)

is the same as

session.query(SomeModel.col1)

for alias, we can use .label()

session.query(SomeModel.col1.label('some alias name'))

回答 2

您可以使用load_only函数:

from sqlalchemy.orm import load_only

fields = ['name', 'addr', 'phone', 'url']
companies = session.query(SomeModel).options(load_only(*fields)).all()

You can use load_only function:

from sqlalchemy.orm import load_only

fields = ['name', 'addr', 'phone', 'url']
companies = session.query(SomeModel).options(load_only(*fields)).all()

回答 3

您可以使用Model.query,因为Model(通常是它的基类,尤其是在使用声明性扩展的情况下)已分配Sesssion.query_property。在这种情况下,Model.query相当于Session.query(Model)

我不知道修改查询返回的列的方法(除非使用添加更多的方法add_columns())。
因此,最好的选择是使用Session.query(Model.col1, Model.col2, ...)(如Salil所示)。

You can use Model.query, because the Model (or usually its base class, especially in cases where declarative extension is used) is assigned Sesssion.query_property. In this case the Model.query is equivalent to Session.query(Model).

I am not aware of the way to modify the columns returned by the query (except by adding more using add_columns()).
So your best shot is to use the Session.query(Model.col1, Model.col2, ...) (as already shown by Salil).


回答 4

您可以使用Query.values,Query.values

session.query(SomeModel).values('id', 'user')

You can use Query.values, Query.values

session.query(SomeModel).values('id', 'user')


回答 5

这里的一个例子:

movies = Movie.query.filter(Movie.rating != 0).order_by(desc(Movie.rating)).all()

我在数据库中查询评级为<> 0的电影,然后首先按最高评级对它们进行评级。

在这里看看:在Flask-SQLAlchemy中选择,插入,删除

An example here:

movies = Movie.query.filter(Movie.rating != 0).order_by(desc(Movie.rating)).all()

I query the db for movies with rating <> 0, and then I order them by rating with the higest rating first.

Take a look here: Select, Insert, Delete in Flask-SQLAlchemy


ImportError:没有名为MySQLdb的模块

问题:ImportError:没有名为MySQLdb的模块

我指的是以下教程来为我的Web应用程序创建登录页面。 http://code.tutsplus.com/tutorials/intro-to-flask-signing-in-and-out–net-29982

我的数据库有问题。我正在

ImportError: No module named MySQLdb

当我执行

http://127.0.0.1:5000/testdb

我已经尝试了所有可能的方法来安装python mysql,这是本教程中提到的一种,easy_install,sudo apt-get install。

我已经在虚拟环境中安装了mysql。我的目录结构与本教程中说明的目录结构相同。该模块已成功安装在我的系统中,但仍然出现此错误。

请帮忙。是什么原因造成的。

I am referring the following tutorial to make a login page for my web application. http://code.tutsplus.com/tutorials/intro-to-flask-signing-in-and-out–net-29982

I am having issue with the database. I am getting an

ImportError: No module named MySQLdb

when I execute

http://127.0.0.1:5000/testdb

I have tried all possible ways to install python mysql, the one mentioned in the tutorial, easy_install, sudo apt-get install.

I have installed mysql in my virtual env. My directory structure is just the same as whats explained in the tutorial. The module is sucessfully installed in my system and still I am getting this error.

Please help. What could be causing this.


回答 0

如果您在编译二进制扩展名时遇到问题,或者在无法扩展的平台上,则可以尝试使用纯python PyMySQL绑定。

只需pip install pymysql切换您的SQLAlchemy URI即可,如下所示:

SQLALCHEMY_DATABASE_URI = 'mysql+pymysql://.....'

您还可以尝试其他一些驱动程序

If you’re having issues compiling the binary extension, or on a platform where you cant, you can try using the pure python PyMySQL bindings.

Simply pip install pymysql and switch your SQLAlchemy URI to start like this:

SQLALCHEMY_DATABASE_URI = 'mysql+pymysql://.....'

There are some other drivers you could also try.


回答 1

或尝试以下方法:

apt-get install python-mysqldb

Or try this:

apt-get install python-mysqldb

回答 2

你可以尝试

pip install mysqlclient

may you try

pip install mysqlclient

回答 3

我的问题是:

return __import__('MySQLdb')
ImportError: No module named MySQLdb

和我的决议:

pip install MySQL-python
yum install mysql-devel.x86_64

在开始的时候,我刚刚安装了MySQL-python,但是问题仍然存在。因此,我认为如果发生此问题,您还应该考虑mysql-devel。希望这可以帮助。

My issue is :

return __import__('MySQLdb')
ImportError: No module named MySQLdb

and my resolution :

pip install MySQL-python
yum install mysql-devel.x86_64

at the very beginning, i just installed MySQL-python, but the issue still existed. So i think if this issue happened, you should also take mysql-devel into consideration. Hope this helps.


回答 4

在研究SQLAlchemy时遇到了这个问题。SQLAlchemy用于MySQL的默认方言是mysql+mysqldb

engine = create_engine('mysql+mysqldb://scott:tiger@localhost/foo')

No module named MySQLdb执行上述命令时出现“ ”错误。要修复它,我安装了mysql-python模块,此问题已解决。

sudo pip install mysql-python

I got this issue when I was working on SQLAlchemy. The default dialect used by SQLAlchemy for MySQL is mysql+mysqldb.

engine = create_engine('mysql+mysqldb://scott:tiger@localhost/foo')

I got the “No module named MySQLdb” error when the above command was executed. To fix it I installed the mysql-python module and the issue was fixed.

sudo pip install mysql-python

回答 5

根据我的经验,它也取决于Python版本。

如果您使用的是Python 3,则@DazWorrall答案对我来说效果很好。

但是,如果您使用的是Python 2,则应该

sudo pip install mysql-python

这将安装“ MySQLdb”模块,而无需更改SQLAlchemy URI。

It depends on Python Version as well in my experience.

If you are using Python 3, @DazWorrall answer worked fine for me.

However, if you are using Python 2, you should

sudo pip install mysql-python

which would install ‘MySQLdb’ module without having to change the SQLAlchemy URI.


回答 6

所以我花了大约5个小时试图弄清楚在尝试运行时如何处理此问题

./manage.py makemigrations

使用Ubuntu Server LTS 16.1,完整的LAMP堆栈,Apache2 MySql 5.7 PHP 7 Python 3和Django 1.10.2,我确实很难找到一个好的答案。实际上,我仍然不满意,但是对我有用的唯一解决方案是……

sudo apt-get install build-essential python-dev libapache2-mod-wsgi-py3 libmysqlclient-dev

其次(从虚拟环境内部)

pip install mysqlclient

我真的不喜欢在尝试设置新的Web服务器时必须使用dev install,但是不幸的是,这种配置是我唯一可以采用的舒适方式。

So I spent about 5 hours trying to figure out how to deal with this issue when trying to run

./manage.py makemigrations

With Ubuntu Server LTS 16.1, a full LAMP stack, Apache2 MySql 5.7 PHP 7 Python 3 and Django 1.10.2 I really struggled to find a good answer to this. In fact, I am still not satisfied, but the ONLY solution that worked for me is this…

sudo apt-get install build-essential python-dev libapache2-mod-wsgi-py3 libmysqlclient-dev

followed by (from inside the virtual environment)

pip install mysqlclient

I really dislike having to use dev installs when I am trying to set up a new web server, but unfortunately this configuration was the only mostly comfortable path I could take.


回答 7

尽管@Edward van Kuik答案是正确的,但并未考虑virtualenv v1.7及更高版本的问题

特别是在Ubuntu 上python-mysqldb通过via apt进行安装时,将其放在下/usr/lib/pythonX.Y/dist-packages,但virtualenv的默认情况下不包含此路径。sys.path

因此,要解决此问题,您应该通过运行类似以下内容的系统包来创建您的virtualenv:

virtualenv --system-site-packages .venv

While @Edward van Kuik‘s answer is correct, it doesn’t take into account an issue with virtualenv v1.7 and above.

In particular installing python-mysqldb via apt on Ubuntu put it under /usr/lib/pythonX.Y/dist-packages, but this path isn’t included by default in the virtualenv’s sys.path.

So to resolve this, you should create your virtualenv with system packages by running something like:

virtualenv --system-site-packages .venv


回答 8

有太多与权限有关的错误,什么也没有。您可能想试试这个:

xcode-select --install

Got so many errors related to permissions and what not. You may wanna try this :

xcode-select --install

回答 9

yum install MySQL-python.x86_64

为我工作。

yum install MySQL-python.x86_64

worked for me.


回答 10

在ubuntu 20中,您可以尝试以下操作:

sudo apt-get install libmysqlclient-dev
sudo apt-get install gcc
pip install mysqlclient

In ubuntu 20 , you can try this :

sudo apt-get install libmysqlclient-dev
sudo apt-get install gcc
pip install mysqlclient

Flask-SQLalchemy更新行的信息

问题:Flask-SQLalchemy更新行的信息

如何更新行的信息?

例如,我想更改ID为5的行的名称列。

How can I update a row’s information?

For example I’d like to alter the name column of the row that has the id 5.


回答 0

使用Flask-SQLAlchemy文档中显示教程来检索对象。拥有要更改的实体后,请更改实体本身。然后,db.session.commit()

例如:

admin = User.query.filter_by(username='admin').first()
admin.email = 'my_new_email@example.com'
db.session.commit()

user = User.query.get(5)
user.name = 'New Name'
db.session.commit()

Flask-SQLAlchemy基于SQLAlchemy,因此请务必同时查看SQLAlchemy文档

Retrieve an object using the tutorial shown in the Flask-SQLAlchemy documentation. Once you have the entity that you want to change, change the entity itself. Then, db.session.commit().

For example:

admin = User.query.filter_by(username='admin').first()
admin.email = 'my_new_email@example.com'
db.session.commit()

user = User.query.get(5)
user.name = 'New Name'
db.session.commit()

Flask-SQLAlchemy is based on SQLAlchemy, so be sure to check out the SQLAlchemy Docs as well.


回答 1

updateSQLAlchemy中的BaseQuery对象有一个方法,由返回filter_by

admin = User.query.filter_by(username='admin').update(dict(email='my_new_email@example.com')))
db.session.commit()

update当要更新的对象很多时,使用更改实体的优势就来了。

如果您想add_user授予所有admins 权限,

rows_changed = User.query.filter_by(role='admin').update(dict(permission='add_user'))
db.session.commit()

注意,filter_by使用关键字参数(仅使用一个=),而不是filter使用表达式。

There is a method update on BaseQuery object in SQLAlchemy, which is returned by filter_by.

admin = User.query.filter_by(username='admin').update(dict(email='my_new_email@example.com')))
db.session.commit()

The advantage of using update over changing the entity comes when there are many objects to be updated.

If you want to give add_user permission to all the admins,

rows_changed = User.query.filter_by(role='admin').update(dict(permission='add_user'))
db.session.commit()

Notice that filter_by takes keyword arguments (use only one =) as opposed to filter which takes an expression.


回答 2

如果您修改模型的腌制属性,这将不起作用。腌制的属性应被替换以触发更新:

from flask import Flask
from flask.ext.sqlalchemy import SQLAlchemy
from pprint import pprint

app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqllite:////tmp/users.db'
db = SQLAlchemy(app)


class User(db.Model):
    id = db.Column(db.Integer, primary_key=True)
    name = db.Column(db.String(80), unique=True)
    data = db.Column(db.PickleType())

    def __init__(self, name, data):
        self.name = name
        self.data = data

    def __repr__(self):
        return '<User %r>' % self.username

db.create_all()

# Create a user.
bob = User('Bob', {})
db.session.add(bob)
db.session.commit()

# Retrieve the row by its name.
bob = User.query.filter_by(name='Bob').first()
pprint(bob.data)  # {}

# Modifying data is ignored.
bob.data['foo'] = 123
db.session.commit()
bob = User.query.filter_by(name='Bob').first()
pprint(bob.data)  # {}

# Replacing data is respected.
bob.data = {'bar': 321}
db.session.commit()
bob = User.query.filter_by(name='Bob').first()
pprint(bob.data)  # {'bar': 321}

# Modifying data is ignored.
bob.data['moo'] = 789
db.session.commit()
bob = User.query.filter_by(name='Bob').first()
pprint(bob.data)  # {'bar': 321}

This does not work if you modify a pickled attribute of the model. Pickled attributes should be replaced in order to trigger updates:

from flask import Flask
from flask.ext.sqlalchemy import SQLAlchemy
from pprint import pprint

app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqllite:////tmp/users.db'
db = SQLAlchemy(app)


class User(db.Model):
    id = db.Column(db.Integer, primary_key=True)
    name = db.Column(db.String(80), unique=True)
    data = db.Column(db.PickleType())

    def __init__(self, name, data):
        self.name = name
        self.data = data

    def __repr__(self):
        return '<User %r>' % self.username

db.create_all()

# Create a user.
bob = User('Bob', {})
db.session.add(bob)
db.session.commit()

# Retrieve the row by its name.
bob = User.query.filter_by(name='Bob').first()
pprint(bob.data)  # {}

# Modifying data is ignored.
bob.data['foo'] = 123
db.session.commit()
bob = User.query.filter_by(name='Bob').first()
pprint(bob.data)  # {}

# Replacing data is respected.
bob.data = {'bar': 321}
db.session.commit()
bob = User.query.filter_by(name='Bob').first()
pprint(bob.data)  # {'bar': 321}

# Modifying data is ignored.
bob.data['moo'] = 789
db.session.commit()
bob = User.query.filter_by(name='Bob').first()
pprint(bob.data)  # {'bar': 321}

回答 3

仅分配值并提交它们将适用于除JSON和Pickled属性以外的所有数据类型。由于上面已经解释了腌制类型,因此我将记下更新JSON的方法略有不同但很简单。

class User(db.Model):
    id = db.Column(db.Integer, primary_key=True)
    name = db.Column(db.String(80), unique=True)
    data = db.Column(db.JSON)

def __init__(self, name, data):
    self.name = name
    self.data = data

假设模型如上。

user = User("Jon Dove", {"country":"Sri Lanka"})
db.session.add(user)
db.session.flush()
db.session.commit()

这会将用户与数据{“ country”:“ Sri Lanka”}一起添加到MySQL数据库中

修改数据将被忽略。我的无效代码如下。

user = User.query().filter(User.name=='Jon Dove')
data = user.data
data["province"] = "south"
user.data = data
db.session.merge(user)
db.session.flush()
db.session.commit()

与其完成将JSON复制到新字典(而不是像上面那样将其分配给新变量)的繁琐工作,我应该找到了一种简单的方法来完成该工作。有一种方法可以标记JSON已更改的系统。

以下是工作代码。

from sqlalchemy.orm.attributes import flag_modified
user = User.query().filter(User.name=='Jon Dove')
data = user.data
data["province"] = "south"
user.data = data
flag_modified(user, "data")
db.session.merge(user)
db.session.flush()
db.session.commit()

这就像一个魅力。这里还建议了另一种方法 希望对我有帮助。

Just assigning the value and committing them will work for all the data types but JSON and Pickled attributes. Since pickled type is explained above I’ll note down a slightly different but easy way to update JSONs.

class User(db.Model):
    id = db.Column(db.Integer, primary_key=True)
    name = db.Column(db.String(80), unique=True)
    data = db.Column(db.JSON)

def __init__(self, name, data):
    self.name = name
    self.data = data

Let’s say the model is like above.

user = User("Jon Dove", {"country":"Sri Lanka"})
db.session.add(user)
db.session.flush()
db.session.commit()

This will add the user into the MySQL database with data {“country”:”Sri Lanka”}

Modifying data will be ignored. My code that didn’t work is as follows.

user = User.query().filter(User.name=='Jon Dove')
data = user.data
data["province"] = "south"
user.data = data
db.session.merge(user)
db.session.flush()
db.session.commit()

Instead of going through the painful work of copying the JSON to a new dict (not assigning it to a new variable as above), which should have worked I found a simple way to do that. There is a way to flag the system that JSONs have changed.

Following is the working code.

from sqlalchemy.orm.attributes import flag_modified
user = User.query().filter(User.name=='Jon Dove')
data = user.data
data["province"] = "south"
user.data = data
flag_modified(user, "data")
db.session.merge(user)
db.session.flush()
db.session.commit()

This worked like a charm. There is another method proposed along with this method here Hope I’ve helped some one.