Cron和virtualenv

问题:Cron和virtualenv

我正在尝试从cron运行Django管理命令。我正在使用virtualenv使我的项目沙盒化。

我在这里和其他地方都看到了一些示例,这些示例显示了从virtualenv内部运行的管理命令,例如:

0 3 * * * source /home/user/project/env/bin/activate && /home/user/project/manage.py command arg

但是,即使syslog在任务应该启动时显示一个条目,该任务也不会实际运行(脚本的日志文件为空)。如果我从外壳程序手动运行该行,它将按预期工作。

目前,我可以使命令通过cron运行的唯一方法是将这些命令分解并放在一个笨拙的bash包装器脚本中:

#!/bin/sh
source /home/user/project/env/bin/activate
cd /home/user/project/
./manage.py command arg

编辑:

ars提出了一种有效的命令组合:

0 3 * * * cd /home/user/project && /home/user/project/env/bin/python /home/user/project/manage.py command arg

至少就我而言,为virtualenv调用激活脚本没有任何作用。这是可行的,因此在演出中如此。

I am trying to run a Django management command from cron. I am using virtualenv to keep my project sandboxed.

I have seen examples here and elsewhere that show running management commands from within virtualenv’s like:

0 3 * * * source /home/user/project/env/bin/activate && /home/user/project/manage.py command arg

However, even though syslog shows an entry when the task should have started, this task never actually runs (the log file for the script is empty). If I run the line manually from the shell, it works as expected.

The only way I can currently get the command to run via cron, is to break the commands up and put them in a dumb bash wrapper script:

#!/bin/sh
source /home/user/project/env/bin/activate
cd /home/user/project/
./manage.py command arg

EDIT:

ars came up with a working combination of commands:

0 3 * * * cd /home/user/project && /home/user/project/env/bin/python /home/user/project/manage.py command arg

At least in my case, invoking the activate script for the virtualenv did nothing. This works, so on with the show.


回答 0

您应该可以通过python在虚拟环境中使用来执行此操作:

/home/my/virtual/bin/python /home/my/project/manage.py command arg

编辑:如果您的django项目不在PYTHONPATH中,那么您需要切换到正确的目录:

cd /home/my/project && /home/my/virtual/bin/python ...

您也可以尝试从cron记录故障:

cd /home/my/project && /home/my/virtual/bin/python /home/my/project/manage.py > /tmp/cronlog.txt 2>&1

另一件事是尝试在manage.py脚本的最顶部进行相同的更改:

#!/home/my/virtual/bin/python

You should be able to do this by using the python in your virtual environment:

/home/my/virtual/bin/python /home/my/project/manage.py command arg

EDIT: If your django project isn’t in the PYTHONPATH, then you’ll need to switch to the right directory:

cd /home/my/project && /home/my/virtual/bin/python ...

You can also try to log the failure from cron:

cd /home/my/project && /home/my/virtual/bin/python /home/my/project/manage.py > /tmp/cronlog.txt 2>&1

Another thing to try is to make the same change in your manage.py script at the very top:

#!/home/my/virtual/bin/python

回答 1

source从cronfile 运行将无法正常运行,因为cron /bin/sh用作其默认外壳程序,该外壳程序不支持source。您需要将SHELL环境变量设置为/bin/bash

SHELL=/bin/bash
*/10 * * * * root source /path/to/virtualenv/bin/activate && /path/to/build/manage.py some_command > /dev/null

很难找到失败原因,因为/var/log/syslog它不会记录错误详细信息。最好将自己别名为root,以便通过电子邮件发送任何cron错误。只需添加自己即可/etc/aliases运行sendmail -bi

此处提供更多信息:http : //codeinthehole.com/archives/43-Running-django-cronjobs-within-a-virtualenv.html

上面的链接已更改为:https : //codeinthehole.com/tips/running-django-cronjobs-within-a-virtualenv/

Running source from a cronfile won’t work as cron uses /bin/sh as its default shell, which doesn’t support source. You need to set the SHELL environment variable to be /bin/bash:

SHELL=/bin/bash
*/10 * * * * root source /path/to/virtualenv/bin/activate && /path/to/build/manage.py some_command > /dev/null

It’s tricky to spot why this fails as /var/log/syslog doesn’t log the error details. Best to alias yourself to root so you get emailed with any cron errors. Simply add yourself to /etc/aliases and run sendmail -bi.

More info here: http://codeinthehole.com/archives/43-Running-django-cronjobs-within-a-virtualenv.html

the link above is changed to: https://codeinthehole.com/tips/running-django-cronjobs-within-a-virtualenv/


回答 2

不要再看了:

0 3 * * * /usr/bin/env bash -c 'cd /home/user/project && source /home/user/project/env/bin/activate && ./manage.py command arg' > /dev/null 2>&1

通用方法:

* * * * * /usr/bin/env bash -c 'YOUR_COMMAND_HERE' > /dev/null 2>&1

这样做的好处是您无需将SHELLcrontab 的变量从更改shbash

Don’t look any further:

0 3 * * * /usr/bin/env bash -c 'cd /home/user/project && source /home/user/project/env/bin/activate && ./manage.py command arg' > /dev/null 2>&1

Generic approach:

* * * * * /usr/bin/env bash -c 'YOUR_COMMAND_HERE' > /dev/null 2>&1

The beauty about this is you DO NOT need to change the SHELL variable for crontab from sh to bash


回答 3

使用virtualenv时,运行python cron作业的唯一正确方法是激活环境,然后执行环境的python来运行代码。

一种方法是activate_this在您的python脚本中使用virtualenv ,请参阅:http : //virtualenv.readthedocs.org/en/latest/userguide.html#using-virtualenv-without-bin-python

另一个解决方案是回显完整的命令,包括激活环境并将其传递到/bin/bash。考虑为您的/etc/crontab

***** root echo 'source /env/bin/activate; python /your/script' | /bin/bash

The only correct way to run python cron jobs when using a virtualenv is to activate the environment and then execute the environment’s python to run your code.

One way to do this is use virtualenv’s activate_this in your python script, see: http://virtualenv.readthedocs.org/en/latest/userguide.html#using-virtualenv-without-bin-python

Another solution is echoing the complete command including activating the environment and piping it into /bin/bash. Consider this for your /etc/crontab:

***** root echo 'source /env/bin/activate; python /your/script' | /bin/bash

回答 4

与其摆弄特定于virtualenv的shebang,不如摆PATH在crontab上。

在激活的virtualenv中,运行以下三个命令,并且python脚本应该可以正常工作:

$ echo "PATH=$PATH" > myserver.cron
$ crontab -l >> myserver.cron
$ crontab myserver.cron

现在,crontab的第一行应如下所示:

PATH=/home/me/virtualenv/bin:/usr/bin:/bin:  # [etc...]

Rather than mucking around with virtualenv-specific shebangs, just prepend PATH onto the crontab.

From an activated virtualenv, run these three commands and python scripts should just work:

$ echo "PATH=$PATH" > myserver.cron
$ crontab -l >> myserver.cron
$ crontab myserver.cron

The crontab’s first line should now look like this:

PATH=/home/me/virtualenv/bin:/usr/bin:/bin:  # [etc...]

回答 5

对我来说最好的解决方案是

  • 使用venv bin /目录中的python二进制文件
  • 设置python路径以包含venv modules目录。

man python提到使用以下命令在shell中$PYTHONPATH或在python 中修改路径sys.path

其他答案提到使用shell进行此操作的想法。在python中,将以下行添加到我的脚本中使我可以直接从cron成功运行它。

import sys
sys.path.insert(0,'/path/to/venv/lib/python3.3/site-packages');

这是交互式会话中的外观-

Python 3.3.2+ (default, Feb 28 2014, 00:52:16) 
[GCC 4.8.1] on linux
Type "help", "copyright", "credits" or "license" for more information.

>>> import sys

>>> sys.path
['', '/usr/lib/python3.3', '/usr/lib/python3.3/plat-x86_64-linux-gnu', '/usr/lib/python3.3/lib-dynload']

>>> import requests
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named 'requests'   

>>> sys.path.insert(0,'/path/to/venv/modules/');

>>> import requests
>>>

The best solution for me was to both

  • use the python binary in the venv bin/ directory
  • set the python path to include the venv modules directory.

man python mentions modifying the path in shell at $PYTHONPATH or in python with sys.path

Other answers mention ideas for doing this using the shell. From python, adding the following lines to my script allows me to successfully run it directly from cron.

import sys
sys.path.insert(0,'/path/to/venv/lib/python3.3/site-packages');

Here’s how it looks in an interactive session —

Python 3.3.2+ (default, Feb 28 2014, 00:52:16) 
[GCC 4.8.1] on linux
Type "help", "copyright", "credits" or "license" for more information.

>>> import sys

>>> sys.path
['', '/usr/lib/python3.3', '/usr/lib/python3.3/plat-x86_64-linux-gnu', '/usr/lib/python3.3/lib-dynload']

>>> import requests
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named 'requests'   

>>> sys.path.insert(0,'/path/to/venv/modules/');

>>> import requests
>>>

回答 6

我想添加这个内容是因为我花了一些时间解决问题,但在这里找不到cron和virtualenv中变量使用组合的答案。所以也许会帮助某人。

PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
DIR_SMTH="cd /smth"
VENV=". venv/bin/activate"
CMD="some_python_bin do_something"
# m h  dom mon dow   command
0 * * * * $DIR_SMTH && $VENV && $CMD -k2 some_target >> /tmp/crontest.log 2>&1

像这样配置时效果不佳

DIR_SMTH =“ cd / smth &&。venv / bin / activate”

感谢@ davidwinterbottom@ reed-sandberg@mkb提供正确的方向。在您的python需要运行一个脚本(该脚本必须从venv / bin目录运行另一个python二进制文件)之前,可接受的答案实际上可以正常工作。

I’d like to add this because I spent some time solving the issue and did not find an answer here for combination of variables usage in cron and virtualenv. So maybe it’ll help someone.

PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
DIR_SMTH="cd /smth"
VENV=". venv/bin/activate"
CMD="some_python_bin do_something"
# m h  dom mon dow   command
0 * * * * $DIR_SMTH && $VENV && $CMD -k2 some_target >> /tmp/crontest.log 2>&1

It did not work well when it was configured like

DIR_SMTH=”cd /smth && . venv/bin/activate”

Thanks @davidwinterbottom, @reed-sandberg and @mkb for giving the right direction. The accepted answer actually works fine until your python need to run a script which have to run another python binary from venv/bin directory.


回答 7

这是一个对我有效的解决方案。

source /root/miniconda3/etc/profile.d/conda.sh && \
conda activate <your_env> && \
python <your_application> &

我在Ubuntu 18.04.3 LTS上使用带有Conda版本4.7.12的miniconda。

我可以将以上内容放入脚本中,并通过crontab进行运行,而不会遇到任何麻烦。

This is a solution that has worked well for me.

source /root/miniconda3/etc/profile.d/conda.sh && \
conda activate <your_env> && \
python <your_application> &

I am using miniconda with Conda version 4.7.12 on a Ubuntu 18.04.3 LTS.

I am able to place the above inside a script and run it via crontab as well without any trouble.


回答 8

python脚本

from datetime import datetime                                                                                                                                                                
import boto   # check wheather its taking the virtualenv or not                                                                                                                                                                        
import sys                                                                                                                                                                                   
param1=sys.argv[1]     #Param                                                                                                                                                                                                                                                                                                                                                                    
myFile = open('appendtxt.txt', 'a')                                                                                                                                                      
myFile.write('\nAccessed on ' + param1+str(datetime.now())) 

Cron命令

 */1 * * * *  cd /Workspace/testcron/ && /Workspace/testcron/venvcron/bin/python3  /Workspace/testcron/testcronwithparam.py param  

在上面的命令中

  • * / 1 * * * * -每1分钟执行一次
  • cd / Workspace / testcron /-python脚本的路径
  • / Workspace / testcron / venvcron / bin / python3 -Virtualenv路径
  • Workspace / testcron / testcronwithparam.py-文件路径
  • 参数 -参数

python script

from datetime import datetime                                                                                                                                                                
import boto   # check wheather its taking the virtualenv or not                                                                                                                                                                        
import sys                                                                                                                                                                                   
param1=sys.argv[1]     #Param                                                                                                                                                                                                                                                                                                                                                                    
myFile = open('appendtxt.txt', 'a')                                                                                                                                                      
myFile.write('\nAccessed on ' + param1+str(datetime.now())) 

Cron command

 */1 * * * *  cd /Workspace/testcron/ && /Workspace/testcron/venvcron/bin/python3  /Workspace/testcron/testcronwithparam.py param  

In above command

  • */1 * * * * – Execute every one minte
  • cd /Workspace/testcron/ – Path of the python script
  • /Workspace/testcron/venvcron/bin/python3 – Virtualenv path
  • Workspace/testcron/testcronwithparam.py – File path
  • param – parameter