Python 实用宝典

Question 1

Node.js is a perfect match for our web project, but there are few computational tasks for which we would prefer Python. We also already have a Python code for them. We are highly concerned about speed, what is the most elegant way how to call a Python “worker” from node.js in an asynchronous non-blocking way?

Question 2

For communication between node.js and Python server, I would use Unix sockets if both processes run on the same server and TCP/IP sockets otherwise. For marshaling protocol I would take JSON or protocol buffer. If threaded Python shows up to be a bottleneck, consider using Twisted Python, which provides the same event driven concurrency as do node.js.

If you feel adventurous, learn clojure (clojurescript, clojure-py) and you’ll get the same language that runs and interoperates with existing code on Java, JavaScript (node.js included), CLR and Python. And you get superb marshalling protocol by simply using clojure data structures.

Question 3

This sounds like a scenario where zeroMQ would be a good fit. It’s a messaging framework that’s similar to using TCP or Unix sockets, but it’s much more robust (http://zguide.zeromq.org/py:all)

There’s a library that uses zeroMQ to provide a RPC framework that works pretty well. It’s called zeroRPC (http://www.zerorpc.io/). Here’s the hello world.

Python “Hello x” server:

import zerorpc

class HelloRPC(object):
    '''pass the method a name, it replies "Hello name!"'''
    def hello(self, name):
        return "Hello, {0}!".format(name)

def main():
    s = zerorpc.Server(HelloRPC())
    s.bind("tcp://*:4242")
    s.run()

if __name__ == "__main__" : main()

And the node.js client:

var zerorpc = require("zerorpc");

var client = new zerorpc.Client();
client.connect("tcp://127.0.0.1:4242");
//calls the method on the python object
client.invoke("hello", "World", function(error, reply, streaming) {
    if(error){
        console.log("ERROR: ", error);
    }
    console.log(reply);
});

Or vice-versa, node.js server:

var zerorpc = require("zerorpc");

var server = new zerorpc.Server({
    hello: function(name, reply) {
        reply(null, "Hello, " + name, false);
    }
});

server.bind("tcp://0.0.0.0:4242");

And the python client

import zerorpc, sys

c = zerorpc.Client()
c.connect("tcp://127.0.0.1:4242")
name = sys.argv[1] if len(sys.argv) > 1 else "dude"
print c.hello(name)

Question 4

If you arrange to have your Python worker in a separate process (either long-running server-type process or a spawned child on demand), your communication with it will be asynchronous on the node.js side. UNIX/TCP sockets and stdin/out/err communication are inherently async in node.

Question 5

I’d consider also Apache Thrift http://thrift.apache.org/

It can bridge between several programming languages, is highly efficient and has support for async or sync calls. See full features here http://thrift.apache.org/docs/features/

The multi language can be useful for future plans, for example if you later want to do part of the computational task in C++ it’s very easy to do add it to the mix using Thrift.

Question 6

I’ve had a lot of success using thoonk.js along with thoonk.py. Thoonk leverages Redis (in-memory key-value store) to give you feed (think publish/subscribe), queue and job patterns for communication.

Why is this better than unix sockets or direct tcp sockets? Overall performance may be decreased a little, however Thoonk provides a really simple API that simplifies having to manually deal with a socket. Thoonk also helps make it really trivial to implement a distributed computing model that allows you to scale your python workers to increase performance, since you just spin up new instances of your python workers and connect them to the same redis server.

Question 7

I’d recommend using some work queue using, for example, the excellent Gearman, which will provide you with a great way to dispatch background jobs, and asynchronously get their result once they’re processed.

The advantage of this, used heavily at Digg (among many others) is that it provides a strong, scalable and robust way to make workers in any language to speak with clients in any language.

Question 8

Update 2019

There are several ways to achieve this and here is the list in increasing order of complexity

Python Shell, you will write streams to the python console and it will write back to you
Redis Pub Sub, you can have a channel listening in Python while your node js publisher pushes data
Websocket connection where Node acts as the client and Python acts as the server or vice-versa
API connection with Express/Flask/Tornado etc working separately with an API endpoint exposed for the other to query

Approach 1 Python Shell Simplest approach

source.js file

const ps = require('python-shell')
// very important to add -u option since our python script runs infinitely
var options = {
    pythonPath: '/Users/zup/.local/share/virtualenvs/python_shell_test-TJN5lQez/bin/python',
    pythonOptions: ['-u'], // get print results in real-time
    // make sure you use an absolute path for scriptPath
    scriptPath: "./subscriber/",
    // args: ['value1', 'value2', 'value3'],
    mode: 'json'
};

const shell = new ps.PythonShell("destination.py", options);

function generateArray() {
    const list = []
    for (let i = 0; i < 1000; i++) {
        list.push(Math.random() * 1000)
    }
    return list
}

setInterval(() => {
    shell.send(generateArray())
}, 1000);

shell.on("message", message => {
    console.log(message);
})

destination.py file

import datetime
import sys
import time
import numpy
import talib
import timeit
import json
import logging
logging.basicConfig(format='%(asctime)s : %(levelname)s : %(message)s', level=logging.INFO)

size = 1000
p = 100
o = numpy.random.random(size)
h = numpy.random.random(size)
l = numpy.random.random(size)
c = numpy.random.random(size)
v = numpy.random.random(size)

def get_indicators(values):
    # Return the RSI of the values sent from node.js
    numpy_values = numpy.array(values, dtype=numpy.double) 
    return talib.func.RSI(numpy_values, 14)

for line in sys.stdin:
    l = json.loads(line)
    print(get_indicators(l))
    # Without this step the output may not be immediately available in node
    sys.stdout.flush()

Notes: Make a folder called subscriber which is at the same level as source.js file and put destination.py inside it. Dont forget to change your virtualenv environment

Question 9

I have terminal access to a VPS running centos 5.9 and default python 2.4.3 installed. I also installed python 2.7.3 via these commands: (I used make altinstall instead of make install)

wget http://www.python.org/ftp/python/2.7.3/Python-2.7.3.tgz
tar -xf Python-2.7.3.tgz
cd Python-2.7.3
./configure
make
make altinstall

then I installed node.js from source via these commands:

python2.7 ./configure
make
make install

The problem is, when I use npm install and try to install a node.js package which requires python > 2.4.3 I get this error:

gyp ERR! configure error
gyp ERR! stack Error: Python executable "python" is v2.4.3, which is not supported by gyp.
gyp ERR! stack You can pass the --python switch to point to Python >= v2.5.0 & < 3.0.0.
gyp ERR! stack     at failPythonVersion (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/configure.js:125:14)
gyp ERR! stack     at /usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/configure.js:114:9

how should I “pass the –python switch to point to Python >= v2.5.0”?

Question 10

You can use --python option to npm like so:

npm install --python=python2.7

or set it to be used always:

npm config set python python2.7

Npm will in turn pass this option to node-gyp when needed.

(note: I’m the one who opened an issue on Github to have this included in the docs, as there were so many questions about it ;-) )

Question 11

set python to python2.7 before running npm install

Linux:

export PYTHON=python2.7

Windows:

set PYTHON=python2.7

Question 12

For Windows users something like this should work:

PS C:\angular> npm install --python=C:\Python27\python.exe

Question 13

Ok, so you’ve found a solution already. Just wanted to share what has been useful to me so many times;

I have created setpy2 alias which helps me switch python.

alias setpy2="mkdir -p /tmp/bin; ln -s `which python2.7` /tmp/bin/python; export PATH=/tmp/bin:$PATH"

Execute setpy2 before you run npm install. The switch stays in effect until you quit the terminal, afterwards python is set back to system default.

You can make use of this technique for any other command/tool as well.

Question 14

for quick one time use this works, npm install –python=”c:\python27″

Question 15

This one works better if you don’t have the python on path or want to specify the directory :

//for Windows
npm config set python C:\Python27\python.exe

//for Linux
npm config set python /usr/bin/python27

Question 16

I have an Express Node.js application, but I also have a machine learning algorithm to use in Python. Is there a way I can call Python functions from my Node.js application to make use of the power of machine learning libraries?

Question 17

Easiest way I know of is to use “child_process” package which comes packaged with node.

Then you can do something like:

const spawn = require("child_process").spawn;
const pythonProcess = spawn('python',["path/to/script.py", arg1, arg2, ...]);

Then all you have to do is make sure that you import sys in your python script, and then you can access arg1 using sys.argv[1], arg2 using sys.argv[2], and so on.

To send data back to node just do the following in the python script:

print(dataToSendBack)
sys.stdout.flush()

And then node can listen for data using:

pythonProcess.stdout.on('data', (data) => {
    // Do something with the data returned from python script
});

Since this allows multiple arguments to be passed to a script using spawn, you can restructure a python script so that one of the arguments decides which function to call, and the other argument gets passed to that function, etc.

Hope this was clear. Let me know if something needs clarification.

Question 18

Example for people who are from Python background and want to integrate their machine learning model in the Node.js application:

It uses the child_process core module:

const express = require('express')
const app = express()

app.get('/', (req, res) => {

    const { spawn } = require('child_process');
    const pyProg = spawn('python', ['./../pypy.py']);

    pyProg.stdout.on('data', function(data) {

        console.log(data.toString());
        res.write(data);
        res.end('end');
    });
})

app.listen(4000, () => console.log('Application listening on port 4000!'))

It doesn’t require sys module in your Python script.

Below is a more modular way of performing the task using Promise:

const express = require('express')
const app = express()

let runPy = new Promise(function(success, nosuccess) {

    const { spawn } = require('child_process');
    const pyprog = spawn('python', ['./../pypy.py']);

    pyprog.stdout.on('data', function(data) {

        success(data);
    });

    pyprog.stderr.on('data', (data) => {

        nosuccess(data);
    });
});

app.get('/', (req, res) => {

    res.write('welcome\n');

    runPy.then(function(fromRunpy) {
        console.log(fromRunpy.toString());
        res.end(fromRunpy);
    });
})

app.listen(4000, () => console.log('Application listening on port 4000!'))

Question 19

The python-shell module by extrabacon is a simple way to run Python scripts from Node.js with basic, but efficient inter-process communication and better error handling.

Installation: npm install python-shell.

Running a simple Python script:

var PythonShell = require('python-shell');

PythonShell.run('my_script.py', function (err) {
  if (err) throw err;
  console.log('finished');
});

Running a Python script with arguments and options:

var PythonShell = require('python-shell');

var options = {
  mode: 'text',
  pythonPath: 'path/to/python',
  pythonOptions: ['-u'],
  scriptPath: 'path/to/my/scripts',
  args: ['value1', 'value2', 'value3']
};

PythonShell.run('my_script.py', options, function (err, results) {
  if (err) 
    throw err;
  // Results is an array consisting of messages collected during execution
  console.log('results: %j', results);
});

For the full documentation and source code, check out https://github.com/extrabacon/python-shell

Question 20

You can now use RPC libraries that support Python and Javascript such as zerorpc

From their front page:

Node.js Client

var zerorpc = require("zerorpc");

var client = new zerorpc.Client();
client.connect("tcp://127.0.0.1:4242");

client.invoke("hello", "RPC", function(error, res, more) {
    console.log(res);
});

Python Server

import zerorpc

class HelloRPC(object):
    def hello(self, name):
        return "Hello, %s" % name

s = zerorpc.Server(HelloRPC())
s.bind("tcp://0.0.0.0:4242")
s.run()

Question 21

Most of previous answers call the success of the promise in the on(“data”), it is not the proper way to do it because if you receive a lot of data you will only get the first part. Instead you have to do it on the end event.

const { spawn } = require('child_process');
const pythonDir = (__dirname + "/../pythonCode/"); // Path of python script folder
const python = pythonDir + "pythonEnv/bin/python"; // Path of the Python interpreter

/** remove warning that you don't care about */
function cleanWarning(error) {
    return error.replace(/Detector is not able to detect the language reliably.\n/g,"");
}

function callPython(scriptName, args) {
    return new Promise(function(success, reject) {
        const script = pythonDir + scriptName;
        const pyArgs = [script, JSON.stringify(args) ]
        const pyprog = spawn(python, pyArgs );
        let result = "";
        let resultError = "";
        pyprog.stdout.on('data', function(data) {
            result += data.toString();
        });

        pyprog.stderr.on('data', (data) => {
            resultError += cleanWarning(data.toString());
        });

        pyprog.stdout.on("end", function(){
            if(resultError == "") {
                success(JSON.parse(result));
            }else{
                console.error(`Python error, you can reproduce the error with: \n${python} ${script} ${pyArgs.join(" ")}`);
                const error = new Error(resultError);
                console.error(error);
                reject(resultError);
            }
        })
   });
}
module.exports.callPython = callPython;

Call:

const pythonCaller = require("../core/pythonCaller");
const result = await pythonCaller.callPython("preprocessorSentiment.py", {"thekeyYouwant": value});

python:

try:
    argu = json.loads(sys.argv[1])
except:
    raise Exception("error while loading argument")

Question 22

I’m on node 10 and child process 1.0.2. The data from python is a byte array and has to be converted. Just another quick example of making a http request in python.

node

const process = spawn("python", ["services/request.py", "https://www.google.com"])

return new Promise((resolve, reject) =>{
    process.stdout.on("data", data =>{
        resolve(data.toString()); // <------------ by default converts to utf-8
    })
    process.stderr.on("data", reject)
})

request.py

import urllib.request
import sys

def karl_morrison_is_a_pedant():   
    response = urllib.request.urlopen(sys.argv[1])
    html = response.read()
    print(html)
    sys.stdout.flush()

karl_morrison_is_a_pedant()

p.s. not a contrived example since node’s http module doesn’t load a few requests I need to make

Question 23

You could take your python, transpile it, and then call it as if it were javascript. I have done this succesfully for screeps and even got it to run in the browser a la brython.

Question 24

/*eslint-env es6*/
/*global require*/
/*global console*/
var express = require('express'); 
var app = express();

// Creates a server which runs on port 3000 and  
// can be accessed through localhost:3000
app.listen(3000, function() { 
    console.log('server running on port 3000'); 
} ) 

app.get('/name', function(req, res) {

    console.log('Running');

    // Use child_process.spawn method from  
    // child_process module and assign it 
    // to variable spawn 
    var spawn = require("child_process").spawn;   
    // Parameters passed in spawn - 
    // 1. type_of_script 
    // 2. list containing Path of the script 
    //    and arguments for the script  

    // E.g : http://localhost:3000/name?firstname=Levente
    var process = spawn('python',['apiTest.py', 
                        req.query.firstname]);

    // Takes stdout data from script which executed 
    // with arguments and send this data to res object
    var output = '';
    process.stdout.on('data', function(data) {

        console.log("Sending Info")
        res.end(data.toString('utf8'));
    });

    console.log(output);
});

This worked for me. Your python.exe must be added to you path variables for this code snippet. Also, make sure your python script is in your project folder.

Question 25

In nodejs, I can do npm install package --save-dev to save the installed package into the package.

How do I achieve the same thing in Python package manager pip? I would like to save the package name and its version into, say, requirements.pip just after installing the package using something like pip install package --save-dev requirements.pip.

Question 26

There isn’t an equivalent with pip.

Best way is to pip install package && pip freeze > requirements.txt

You can see all the available options on their documentation page.

If it really bothers you, it wouldn’t be too difficult to write a custom bash script (pips) that takes a -s argument and freezes to your requirements.txt file automatically.

Edit 1

Since writing this there has been no change in providing an auto --save-dev option similar to NPM however Kenneth Reitz (author of requests and many more) has released some more info about a better pip workflow to better handle pip updates.

Edit 2

Linked from the “better pip workflow” article above it is now recommended to use pipenv to manage requirements and virtual environments. Having used this a lot recently I would like to summarise how simple the transition is:

Install pipenv (on Mac)

brew install pipenv

pipenv creates and manages it’s own virtual environments so in a project with an existing requirements.txt, installing all requirements (I use Python3.7 but you can remove the --three if you do not) is as simple as:

pipenv --three install

Activating the virtualenv to run commands is also easy

pipenv shell

Installing requirements will automatically update the Pipfile and Pipfile.lock

pipenv install <package>

It’s also possible to update out-of-date packages

pipenv update

I highly recommend checking it out especially if coming from a npm background as it has a similar feel to package.json and package-lock.json

Question 27

This simple line is a starting point. You can easily built a bash command to reuse the PACKAGE in the line.

pip install PACKAGE && pip freeze | grep PACKAGE >> requirements.txt

Thanks to @devsnd for the simple bash function example:

function pip-install-save { 
    pip install $1 && pip freeze | grep $1 >> requirements.txt
}

To use it, just run:

pip-install-save some-package

Question 28

I’ve created python package that wraps around the actual pip called pipm. All pip commands will work as it is, plus they will be reflected in the requirements file. Unlike pip-save(similar tool I found and wasn’t able to use) it can handle many files and environments(test, dev, production, etc. ). It also has command to upgrade all/any of your dependencies.

installation

pipm install pkg-name

installation as development dependency

pipm install pkg-name --dev

installation as testing dependency

pipm install pkg-name --test

removal

pipm uninstall pkg-name

update all your dependencies

pipm update

install all your dependencies from the requirements file

pipm install

including development dependencies

pipm install --dev

Question 29

Update: apparently, pipenv is not officially endorsed by Python maintainers, and the previously-linked page is owned by a different organization. The tool has its pros and cons, but the below solution still achieves the result that the OP is seeking.

pipenv is a dependency management tool that wraps pip and, among other things, provides what you’re asking:

https://pipenv.kennethreitz.org/en/latest/#example-pipenv-workflow

$ pipenv install <package>

This will create a Pipfile if one doesn’t exist. If one does exist, it will automatically be edited with the new package your provided.

A Pipfile is a direct equivalent of package.json, while Pipfile.lock corresponds to package-lock.json.

Question 30

I made a quick hack on pip to add --save option to install/uninstall commands.

Please have a look at my blog for more information about this hack: http://blog.abhiomkar.in/2015/11/12/pip-save-npm-like-behaviour-to-pip/

Installation (GitHub): https://github.com/abhiomkar/pip-save

Hope this helps.

Question 31

you can manually save it in a Makefile (or a text file and then imported in your Makefile):

PYTHON=.venv/bin/python # path to pyphon
PIP=.venv/bin/pip # path to pip
SOURCE_VENV=. .venv/bin/activate


install:
    virtualenv .venv
    $(SOURCE_VENV) && $(PIP) install -e PACKAGE
    $(SOURCE_VENV) && $(PIP) install -r requirements.txt # other required packages

and then just run make install

Question 32

I am using this small command line to install a package and save its version in requirements.txt : pkg=package && pip install $pkg && echo $(pip freeze | grep -i $pkg) >> requirements.txt

Question 33

How about make a shell function to do this ? Add below code to your ~/.profile or ~/.bashrc

pips() {
    local pkg=$1

    if [ -z "$1" ]; then
        echo "usage: pips <pkg name>"
        return 1
    fi

    local _ins="pip install $pkg"
    eval $_ins
    pip freeze | grep $pkg -i >> requirements.txt
}

then run source ~/.profile or source ~/.bashrc to import it to your current terminal

when you want to install && save a package, just run, for example pips requests. after package was installed, its version will be save into requirements.txt in your current directory.

Question 34

I am getting into a Node.js codebase which requires that I download a few dependencies via NPM, namely jQuery.

In attempting to run npm install jquery, I keep getting this error:

Your environment has been set up for using Node.js 0.8.21 (x64) and NPM

C:\Users\Matt Cashatt>npm install jquery
npm http GET https://registry.npmjs.org/jquery
npm http 304 https://registry.npmjs.org/jquery
npm http GET https://registry.npmjs.org/jsdom
npm http GET https://registry.npmjs.org/xmlhttprequest
npm http GET https://registry.npmjs.org/htmlparser/1.7.6
npm http GET https://registry.npmjs.org/location/0.0.1
npm http GET https://registry.npmjs.org/navigator
npm http GET https://registry.npmjs.org/contextify
npm http 304 https://registry.npmjs.org/htmlparser/1.7.6
npm http 304 https://registry.npmjs.org/xmlhttprequest
npm http 304 https://registry.npmjs.org/location/0.0.1
npm http 304 https://registry.npmjs.org/navigator
npm http 304 https://registry.npmjs.org/jsdom
npm http 304 https://registry.npmjs.org/contextify
npm http GET https://registry.npmjs.org/bindings
npm http GET https://registry.npmjs.org/cssom
npm http GET https://registry.npmjs.org/cssstyle
npm http GET https://registry.npmjs.org/request
npm http 304 https://registry.npmjs.org/bindings

> contextify@0.1.4 install C:\Users\Matt Cashatt\node_modules\jquery\node_module
s\contextify
> node-gyp rebuild


C:\Users\Matt Cashatt\node_modules\jquery\node_modules\contextify>node "C:\Progr
am Files\nodejs\node_modules\npm\bin\node-gyp-bin\\..\..\node_modules\node-gyp\b
in\node-gyp.js" rebuild
npm http 304 https://registry.npmjs.org/cssstyle
npm http 304 https://registry.npmjs.org/cssom
npm http 304 https://registry.npmjs.org/request
gyp ERR! configure error
gyp ERR! stack Error: Can't find Python executable "python", you can set the PYT
HON env variable.
gyp ERR! stack     at failNoPython (C:\Program Files\nodejs\node_modules\npm\nod
e_modules\node-gyp\lib\configure.js:113:14)
gyp ERR! stack     at C:\Program Files\nodejs\node_modules\npm\node_modules\node
-gyp\lib\configure.js:82:11
gyp ERR! stack     at Object.oncomplete (fs.js:297:15)
gyp ERR! System Windows_NT 6.1.7601
gyp ERR! command "node" "C:\\Program Files\\nodejs\\node_modules\\npm\\node_modu
les\\node-gyp\\bin\\node-gyp.js" "rebuild"
gyp ERR! cwd C:\Users\Matt Cashatt\node_modules\jquery\node_modules\contextify
gyp ERR! node -v v0.8.21
gyp ERR! node-gyp -v v0.8.4
gyp ERR! not ok
npm ERR! error rolling back Error: ENOTEMPTY, rmdir 'C:\Users\Matt Cashatt\node_
modules\jquery\node_modules\jsdom\node_modules\request\tests'
npm ERR! error rolling back  jquery@1.8.3 { [Error: ENOTEMPTY, rmdir 'C:\Users\M
att Cashatt\node_modules\jquery\node_modules\jsdom\node_modules\request\tests']
npm ERR! error rolling back   errno: 53,
npm ERR! error rolling back   code: 'ENOTEMPTY',
npm ERR! error rolling back   path: 'C:\\Users\\Matt Cashatt\\node_modules\\jque
ry\\node_modules\\jsdom\\node_modules\\request\\tests' }
npm ERR! contextify@0.1.4 install: `node-gyp rebuild`
npm ERR! `cmd "/c" "node-gyp rebuild"` failed with 1
npm ERR!
npm ERR! Failed at the contextify@0.1.4 install script.
npm ERR! This is most likely a problem with the contextify package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     node-gyp rebuild
npm ERR! You can get their info via:
npm ERR!     npm owner ls contextify
npm ERR! There is likely additional logging output above.

npm ERR! System Windows_NT 6.1.7601
npm ERR! command "C:\\Program Files\\nodejs\\\\node.exe" "C:\\Program Files\\nod
ejs\\node_modules\\npm\\bin\\npm-cli.js" "install" "jquery"
npm ERR! cwd C:\Users\Matt Cashatt
npm ERR! node -v v0.8.21
npm ERR! npm -v 1.2.11
npm ERR! code ELIFECYCLE
npm ERR! Error: ENOENT, lstat 'C:\Users\Matt Cashatt\node_modules\jquery\node_mo
dules\jsdom\node_modules\request\tests\test-pipes.js'
npm ERR! If you need help, you may report this log at:
npm ERR!     <http://github.com/isaacs/npm/issues>
npm ERR! or email it to:
npm ERR!     <npm-@googlegroups.com>

npm ERR! System Windows_NT 6.1.7601
npm ERR! command "C:\\Program Files\\nodejs\\\\node.exe" "C:\\Program Files\\nod
ejs\\node_modules\\npm\\bin\\npm-cli.js" "install" "jquery"
npm ERR! cwd C:\Users\Matt Cashatt
npm ERR! node -v v0.8.21
npm ERR! npm -v 1.2.11
npm ERR! path C:\Users\Matt Cashatt\node_modules\jquery\node_modules\jsdom\node_
modules\request\tests\test-pipes.js
npm ERR! fstream_path C:\Users\Matt Cashatt\node_modules\jquery\node_modules\jsd
om\node_modules\request\tests\test-pipes.js
npm ERR! fstream_type File
npm ERR! fstream_class FileWriter
npm ERR! code ENOENT
npm ERR! errno 34
npm ERR! fstream_stack C:\Program Files\nodejs\node_modules\npm\node_modules\fst
ream\lib\writer.js:284:26
npm ERR! fstream_stack Object.oncomplete (fs.js:297:15)
npm ERR!
npm ERR! Additional logging details can be found in:
npm ERR!     C:\Users\Matt Cashatt\npm-debug.log
npm ERR! not ok code 0

C:\Users\Matt Cashatt>

It looks like the failure is due to a missing Python installation. Well, I have installed Python, set the variable, and rebooted and still the error.

Any clue as to what I am missing?

Question 35

Your problem is that you didn’t set the environment variable.

The error clearly says this:

gyp ERR! stack Error: Can't find Python executable "python", you can set the PYTHON env variable.

And in your comment, you say you did this:

set PYTHONPATH=%PYTHONPATH%;C:\My_python_lib

That’s nice, but that doesn’t set the PYTHON variable, it sets the PYTHONPATH variable.

Meanwhile, just using the set command only affects the current cmd session. If you reboot after that, as you say you did, you end up with a whole new cmd session that doesn’t have that variable set in it.

There are a few ways to set environment variables permanently—the easiest is in the System Control Panel in XP, which is of course different in Vista, different again in 7, and different again in 8, but you can google for it.

Alternatively, just do the set right before the npm command, without rebooting in between.

You can test whether you’ve done things right by doing the exact same thing the config script is trying to do: Before running npm, try running %PYTHON%. If you’ve done it right, you’ll get a Python interpreter (which you can immediately quit). If you get an error, you haven’t done it right.

There are two problems with this:

set PYTHON=%PYTHON%;D:\Python

First, you’re setting PYTHON to ;D:\Python. That extra semicolon is fine for a semicolon-separated list of paths, like PATH or PYTHONPATH, but not for a single value like PYTHON. And likewise, adding a new value to the existing value is what you want when you want to add another path to a list of paths, but not for a single value. So, you just want set PYTHON=D:\Python.

Second, D:\Python is not the path to your Python interpreter. It’s something like D:\Python\Python.exe, or D:\Python\bin\Python.exe. Find the right path, make sure it works on its own (e.g., type D:\Python\bin\Python.exe and make sure you get a Python interpreter), then set the variable and use it.

So:

set PYTHON=D:\Python\bin\Python.exe

Or, if you want to make it permanent, do the equivalent in the Control Panel.

Question 36

If you haven’t got python installed along with all the node-gyp dependencies, simply open Powershell or Git Bash with administrator privileges and execute:

npm install --global --production windows-build-tools

and then to install the package:

npm install --global node-gyp

once installed, you will have all the node-gyp dependencies downloaded, but you still need the environment variable. Validate Python is indeed found in the correct folder:

C:\Users\ben\.windows-build-tools\python27\python.exe

Note – it uses python 2.7 not 3.x as it is not supported

If it doesn’t moan, go ahead and create your (user) environment variable:

setx PYTHON "%USERPROFILE%\.windows-build-tools\python27\python.exe"

restart cmd, and verify the variable exists via set PYTHON which should return the variable

Lastly re-apply npm install <module>

Question 37

For me after installing windows-build-tools with the below comment

npm --add-python-to-path='true' --debug install --global windows-build-tools

running the code below

npm config set python "%USERPROFILE%\.windows-build-tools\python27\python.exe"

has worked.

Question 38

Here is a guide that resolved a lot of these issues for me.

http://www.steveworkman.com/node-js/2012/installing-jsdom-on-windows/

I remember in particular the python version as important. Make sure you install 2.7.3 instead of 3’s.

Question 39

One and/or multiple of those should help:

Add C:\Python27\ to your PATH variable (considering you have Python installed in this directory)
How to set PATH env variable: http://www.computerhope.com/issues/ch000549.htm
Restart your console and/or Windows after setting variable.
In the same section as above (“Environment Variables”), add new variable with name PYTHON and value C:\Python27\python.exe
Restart your console and/or Windows after setting variable.
Open Windows command line (cmd) in Admin mode.
Change directory to your Python installation path: cd C:\Python27
Make symlink needed for some installations: mklink python2.7.exe python.exe

Please note that you should have Python 2.x, NOT 3.x, to run node-gyp based installations!

The text below says about Unix, but Windows version also requires Python 2.x:

You can install with npm:

$ npm install -g node-gyp
You will also need to install:

On Unix:
python (v2.7 recommended, v3.x.x is not supported)
make
A proper C/C++ compiler toolchain, like GCC

This article may also help: https://github.com/nodejs/node-gyp#installation

Question 40

I had the same issue and none of these answers did help. In my case PYTHON variable was set correctly. However python was installed too deep, i.e. has too long path. So, I did the following:

reinstalled python to c:\python
set environmental variable PYTHON to C:\python\python.exe

And that’s it!

Question 41

This helped: https://www.npmjs.com/package/node-gyp

I followed these steps:

npm install -g node-gyp

then:

npm install --global --production windows-build-tools

Question 42

there are some solution to solve this issue : 1 ) run your command prompt as “administrator”.

if first solution doesn’t solve your problem try this one :

2 ) open a command prompt as administrator paste following line of code and hit enter :

npm install --global --production windows-build-tools

Question 43

TL;DR Make a copy or alias of your python.exe with name python2.7.exe

My python 2.7 was installed as

D:\app\Python27\python.exe

I always got this error no matter how I set (and verified) PYTHON env variable:

gyp ERR! stack Error: Can't find Python executable "python2.7", you can set the PYTHON env variable.
gyp ERR! stack     at failNoPython (C:\Program Files\nodejs\node_modules\npm\node_modules\node-gyp\lib\configure.js:103:14)

The reason for this was that in node-gyp’s configure.js the python executable was resolved like:

var python = gyp.opts.python || process.env.PYTHON || 'python'

And it turned out that gyp.opts.python had value ‘python2.7’ thus overriding process.env.PYTHON.

I resolved this by creating an alias for python.exe executable with name node-gyp was looking for:

D:\app\Python27>mklink python2.7.exe python.exe

You need admin rights for this operation.

Question 44

The following worked for me from the command line as admin:

Installing windows-build-tools (this can take 15-20 minutes):

 npm --add-python-to-path='true' --debug install --global windows-build-tools

Adding/updating the environment variable:

setx PYTHON "%USERPROFILE%\.windows-build-tools\python27\python.exe"

Installing node-gyp:

npm install --global node-gyp

Changing the name of the exe file from Python to Python2.7.

C:\Users\username\.windows-build-tools\python27\Python2.7

npm install module_name --save

Question 45

i can’t help but to mention this. If you’re using Python3 and failing with node-gyp, then i’m sad to tell you node-gyp currently doesn’t support python3.

Here is a link for you: https://github.com/nodejs/node-gyp/issues/1268 https://github.com/nodejs/node-gyp/issues/193

Question 46

This is most easiest way to let NPM do everything for you

npm --add-python-to-path='true' --debug install --global windows-build-tools

Question 47

The right way is 1) Download and Install python 2.7.14 from here. 2) Set environment variable for python from here.

done!

note: Please set environment variable accordingly. I answered here for windows.

Question 48

I met the same challenge while trying to install node-sass@4.9.4.

And after looking at the current official documentation, and having read the answers above, i noticed that you might not necessarily have to install node-gyp nor install windows-build tools. This is what it says, here about installing node-gyp on windows. Remember node-gyp is involved in the installation process of node-sass. And you don’t really have to re-install another python version.

This is the savior, configure the python path that “npm” should look for while installing any packages that require build-tools.

C:\> npm config set python /Python36/python

I had installed python3.6.3, on windows-7, there.

Question 49

Why not downloading the python installer here ? It make the work for you when you check the path installation

Question 50

For me, these steps fixed the issue:

1- Running this cmd as admin:

npm install --global --production windows-build-tools

2- Then running npm rebuild after the 1st step is completed (especially completing the python 2.7 installation, which was the main cause of the issue)

Question 51

Here is the correct command: set path=%path%;C:\Python34 [Replace with the correct path of your python installation]

I had the same problem and I just solved this like that.

As some other people pointed out, this is volatile configuration, it only works for the current cmd session, and (obviously) you have to set your path before you run npm install.

I hope this helps.

Question 52

gyp ERR! configure error gyp ERR! stack Error: Can’t find Python executable “python”, you can set the PYT HON env variable.

Not necessary to reinstall, this exception throw by node-gyp script, then try to rebuild. It’s enough setup environment variable like in my case I did:

SET PYTHON=C:\work\_env\Python27\python.exe

Question 53

If you’re trying to use this on Cygwin, then you need to follow the instructions in this answer. (It’s a problem how Cygwin treats Windows symlinks.)

Question 54

Example : pg_config not executable / error node-gyp

Solution : On windows just try to add PATH Env -> C:\Program Files\PostgreSQL\12\bin

Work for me, Now i can use npm i pg-promise for example or other dependencies.

Question 55

For me, The issue was that i was using node’s latest version and not the LTS version which is the stable version and recommended for most users.
Using the LTS Version solved the issue.
You can download from here :

LTS Version

Current Latest Version

问题：结合使用node.js和Python

回答 0

回答 1

回答 2

回答 3

回答 4

回答 5

回答 6

问题：在NPM安装期间如何使用其他版本的python？

回答 0

回答 1

回答 2

回答 3

回答 4

回答 5

问题：如何从Node.js调用Python函数

回答 0

回答 1

回答 2

运行一个简单的Python脚本：

使用参数和选项运行Python脚本：

Running a simple Python script:

Running a Python script with arguments and options:

回答 3

回答 4

回答 5

节点

request.py

node

request.py

回答 6

回答 7

问题：pip等同于`npm install package –save-dev`吗？

回答 0

回答 1

回答 2

安装

安装为开发依赖项

安装为测试依赖项

清除

更新所有依赖

从需求文件安装所有依赖项

包括开发依赖

installation

installation as development dependency

installation as testing dependency

removal

update all your dependencies

install all your dependencies from the requirements file

including development dependencies

回答 3

回答 4

回答 5

回答 6

回答 7

问题：在Windows上运行Python以获取Node.js依赖项

回答 0

回答 1

回答 2

回答 3

回答 4

回答 5

回答 6

回答 7

回答 8

回答 9

回答 10

回答 11

回答 12

回答 13

回答 14

回答 15

回答 16

回答 17

回答 18

回答 19

回答 20

CapRover

这是什么？

我是认真的!谁应该关心CapRover？