从URL获取协议+主机名-Python 实用宝典

从URL获取协议+主机名

Python实用宝典Python实用宝典 知识问答 2021年8月4日
819 0
在我的Django应用中，我需要从引荐来源网址中获取主机名request.META.get('HTTP_REFERER')及其协议，以便从类似以下网址的网址中获取： https://docs.google.com/spreadsheet/ccc?key=blah-blah-blah-blah#gid=1 /programming/1234567/blah-blah-blah-blah http://www.example.com https://www.other-domain.com/whatever/blah/blah/?v1=0&v2=blah+blah ... 我应该得到： https://docs.google.com/ https://stackoverflow.com/ http://www.example.com https://www.other-domain.com/ 我查看了其他相关问题，并找到了有关urlparse的信息，但这并没有成功 >>> urlparse(request.META.get('HTTP_REFERER')).hostname 'docs.google.com'
内容 隐藏
问题：从URL获取协议+主机名
回答 0
回答 1
回答 2
Python3使用urlsplit：
Python3 using urlsplit:
回答 3
回答 4
回答 5
回答 6
回答 7
回答 8
回答 9
回答 10
回答 11
回答 12
回答 13
回答 14

问题：从URL获取协议+主机名

在我的Django应用中，我需要从引荐来源网址中获取主机名request.META.get('HTTP_REFERER')及其协议，以便从类似以下网址的网址中获取：

我应该得到：

我查看了其他相关问题，并找到了有关urlparse的信息，但这并没有成功

>>> urlparse(request.META.get('HTTP_REFERER')).hostname
'docs.google.com'
点击查看英文原文

In my Django app, I need to get the host name from the referrer in request.META.get('HTTP_REFERER') along with its protocol so that from URLs like:

I should get:

I looked over other related questions and found about urlparse, but that didn't do the trick since

>>> urlparse(request.META.get('HTTP_REFERER')).hostname
'docs.google.com'

回答 0

您应该能够做到urlparse（docs：python2，python3）：

from urllib.parse import urlparse
# from urlparse import urlparse  # Python 2
parsed_uri = urlparse('http://stackoverflow.com/questions/1234567/blah-blah-blah-blah' )
result = '{uri.scheme}://{uri.netloc}/'.format(uri=parsed_uri)
print(result)

# gives
'http://stackoverflow.com/'
点击查看英文原文

You should be able to do it with urlparse (docs: python2, python3):

from urllib.parse import urlparse
# from urlparse import urlparse  # Python 2
parsed_uri = urlparse('http://stackoverflow.com/questions/1234567/blah-blah-blah-blah' )
result = '{uri.scheme}://{uri.netloc}/'.format(uri=parsed_uri)
print(result)

# gives
'http://stackoverflow.com/'

回答 1

https://github.com/john-kurkowski/tldextract

这是urlparse的详细版本。它会为您检测域和子域。

从他们的文档中：

>>> import tldextract
>>> tldextract.extract('http://forums.news.cnn.com/')
ExtractResult(subdomain='forums.news', domain='cnn', suffix='com')
>>> tldextract.extract('http://forums.bbc.co.uk/') # United Kingdom
ExtractResult(subdomain='forums', domain='bbc', suffix='co.uk')
>>> tldextract.extract('http://www.worldbank.org.kg/') # Kyrgyzstan
ExtractResult(subdomain='www', domain='worldbank', suffix='org.kg')

ExtractResult 是一个namedtuple，因此可以轻松访问所需的部件。

>>> ext = tldextract.extract('http://forums.bbc.co.uk')
>>> ext.domain
'bbc'
>>> '.'.join(ext[:2]) # rejoin subdomain and domain
'forums.bbc'
点击查看英文原文

https://github.com/john-kurkowski/tldextract

This is a more verbose version of urlparse. It detects domains and subdomains for you.

From their documentation:

>>> import tldextract
>>> tldextract.extract('http://forums.news.cnn.com/')
ExtractResult(subdomain='forums.news', domain='cnn', suffix='com')
>>> tldextract.extract('http://forums.bbc.co.uk/') # United Kingdom
ExtractResult(subdomain='forums', domain='bbc', suffix='co.uk')
>>> tldextract.extract('http://www.worldbank.org.kg/') # Kyrgyzstan
ExtractResult(subdomain='www', domain='worldbank', suffix='org.kg')

ExtractResult is a namedtuple, so it's simple to access the parts you want.

>>> ext = tldextract.extract('http://forums.bbc.co.uk')
>>> ext.domain
'bbc'
>>> '.'.join(ext[:2]) # rejoin subdomain and domain
'forums.bbc'

回答 2

Python3使用urlsplit：

from urllib.parse import urlsplit
url = "http://stackoverflow.com/questions/9626535/get-domain-name-from-url"
base_url = "{0.scheme}://{0.netloc}/".format(urlsplit(url))
print(base_url)
# http://stackoverflow.com/
点击查看英文原文

Python3 using urlsplit:

from urllib.parse import urlsplit
url = "http://stackoverflow.com/questions/9626535/get-domain-name-from-url"
base_url = "{0.scheme}://{0.netloc}/".format(urlsplit(url))
print(base_url)
# http://stackoverflow.com/

回答 3

纯字符串操作:)：

>>> url = "http://stackoverflow.com/questions/9626535/get-domain-name-from-url"
>>> url.split("//")[-1].split("/")[0].split('?')[0]
'stackoverflow.com'
>>> url = "stackoverflow.com/questions/9626535/get-domain-name-from-url"
>>> url.split("//")[-1].split("/")[0].split('?')[0]
'stackoverflow.com'
>>> url = "http://foo.bar?haha/whatever"
>>> url.split("//")[-1].split("/")[0].split('?')[0]
'foo.bar'

就是这样，伙计们。

点击查看英文原文

Pure string operations :):

>>> url = "http://stackoverflow.com/questions/9626535/get-domain-name-from-url"
>>> url.split("//")[-1].split("/")[0].split('?')[0]
'stackoverflow.com'
>>> url = "stackoverflow.com/questions/9626535/get-domain-name-from-url"
>>> url.split("//")[-1].split("/")[0].split('?')[0]
'stackoverflow.com'
>>> url = "http://foo.bar?haha/whatever"
>>> url.split("//")[-1].split("/")[0].split('?')[0]
'foo.bar'

That's all, folks.

回答 4

>>> import urlparse
>>> url = 'http://stackoverflow.com/questions/1234567/blah-blah-blah-blah'
>>> urlparse.urljoin(url, '/')
'http://stackoverflow.com/'
点击查看英文原文 
>>> import urlparse
>>> url = 'http://stackoverflow.com/questions/1234567/blah-blah-blah-blah'
>>> urlparse.urljoin(url, '/')
'http://stackoverflow.com/'

回答 5

如果您认为自己的网址有效，那么它将一直有效

domain = "http://google.com".split("://")[1].split("/")[0]
点击查看英文原文

if you think your url is valid then this will work all the time

domain = "http://google.com".split("://")[1].split("/")[0]

回答 6

纯字符串操作有什么问题吗：

url = 'http://stackoverflow.com/questions/9626535/get-domain-name-from-url'
parts = url.split('//', 1)
print parts[0]+'//'+parts[1].split('/', 1)[0]
>>> http://stackoverflow.com

如果您希望在末尾加上斜杠，请将该脚本扩展如下：

parts = url.split('//', 1)
base = parts[0]+'//'+parts[1].split('/', 1)[0]
print base + (len(url) > len(base) and url[len(base)]=='/'and'/' or '')

可能可以优化一点...

点击查看英文原文

Is there anything wrong with pure string operations:

url = 'http://stackoverflow.com/questions/9626535/get-domain-name-from-url'
parts = url.split('//', 1)
print parts[0]+'//'+parts[1].split('/', 1)[0]
>>> http://stackoverflow.com

If you prefer having a trailing slash appended, extend this script a bit like so:

parts = url.split('//', 1)
base = parts[0]+'//'+parts[1].split('/', 1)[0]
print base + (len(url) > len(base) and url[len(base)]=='/'and'/' or '')

That can probably be optimized a bit ...

回答 7

这是一个稍微改进的版本：

urls = [
    "http://stackoverflow.com:8080/some/folder?test=/questions/9626535/get-domain-name-from-url",
    "Stackoverflow.com:8080/some/folder?test=/questions/9626535/get-domain-name-from-url",
    "http://stackoverflow.com/some/folder?test=/questions/9626535/get-domain-name-from-url",
    "https://StackOverflow.com:8080?test=/questions/9626535/get-domain-name-from-url",
    "stackoverflow.com?test=questions&v=get-domain-name-from-url"]
for url in urls:
    spltAr = url.split("://");
    i = (0,1)[len(spltAr)>1];
    dm = spltAr[i].split("?")[0].split('/')[0].split(':')[0].lower();
    print dm

输出量

stackoverflow.com
stackoverflow.com
stackoverflow.com
stackoverflow.com
stackoverflow.com

小提琴：https ://pyfiddle.io/fiddle/23e4976e-88d2-4757-993e-532aa41b7bf0/ ? i = true

点击查看英文原文

Here is a slightly improved version:

urls = [
    "http://stackoverflow.com:8080/some/folder?test=/questions/9626535/get-domain-name-from-url",
    "Stackoverflow.com:8080/some/folder?test=/questions/9626535/get-domain-name-from-url",
    "http://stackoverflow.com/some/folder?test=/questions/9626535/get-domain-name-from-url",
    "https://StackOverflow.com:8080?test=/questions/9626535/get-domain-name-from-url",
    "stackoverflow.com?test=questions&v=get-domain-name-from-url"]
for url in urls:
    spltAr = url.split("://");
    i = (0,1)[len(spltAr)>1];
    dm = spltAr[i].split("?")[0].split('/')[0].split(':')[0].lower();
    print dm

Output

stackoverflow.com
stackoverflow.com
stackoverflow.com
stackoverflow.com
stackoverflow.com

Fiddle: https://pyfiddle.io/fiddle/23e4976e-88d2-4757-993e-532aa41b7bf0/?i=true

回答 8

这有点钝，但是urlparse在两个方向上都使用：

import urlparse
def uri2schemehostname(uri):
    urlparse.urlunparse(urlparse.urlparse(uri)[:2] + ("",) * 4)

该奇数("",) * 4位是因为urlparse期望精确地 等于len(urlparse.ParseResult._fields)6 的序列

点击查看英文原文

This is a bit obtuse, but uses urlparse in both directions:

import urlparse
def uri2schemehostname(uri):
    urlparse.urlunparse(urlparse.urlparse(uri)[:2] + ("",) * 4)

that odd ("",) * 4 bit is because urlparse expects a sequence of exactly len(urlparse.ParseResult._fields) = 6

回答 9

我知道这是一个老问题，但是今天我也遇到了。用单线解决了这个问题：

import re
result = re.sub(r'(.*://)?([^/?]+).*', '\g<1>\g<2>', url)
点击查看英文原文

I know it's an old question, but I too encountered it today. Solved this with an one-liner:

import re
result = re.sub(r'(.*://)?([^/?]+).*', '\g<1>\g<2>', url)

回答 10

您只需要标准库函数urllib.parse.urlsplit（）。这是Python3的示例：

>>> import urllib.parse
>>> o = urllib.parse.urlsplit('https://user:pass@www.example.com:8080/dir/page.html?q1=test&q2=a2#anchor1')
>>> o.scheme
'https'
>>> o.netloc
'user:pass@www.example.com:8080'
>>> o.hostname
'www.example.com'
>>> o.port
8080
>>> o.path
'/dir/page.html'
>>> o.query
'q1=test&q2=a2'
>>> o.fragment
'anchor1'
>>> o.username
'user'
>>> o.password
'pass'
点击查看英文原文

The standard library function urllib.parse.urlsplit() is all you need. Here is an example for Python3:

>>> import urllib.parse
>>> o = urllib.parse.urlsplit('https://user:pass@www.example.com:8080/dir/page.html?q1=test&q2=a2#anchor1')
>>> o.scheme
'https'
>>> o.netloc
'user:pass@www.example.com:8080'
>>> o.hostname
'www.example.com'
>>> o.port
8080
>>> o.path
'/dir/page.html'
>>> o.query
'q1=test&q2=a2'
>>> o.fragment
'anchor1'
>>> o.username
'user'
>>> o.password
'pass'

回答 11

可以通过re.search（）解决

import re
url = 'https://docs.google.com/spreadsheet/ccc?key=blah-blah-blah-blah#gid=1'
result = re.search(r'^http[s]*:\/\/[\w\.]*', url).group()
print(result)

#result
'https://docs.google.com'
点击查看英文原文

It could be solved by re.search()

import re
url = 'https://docs.google.com/spreadsheet/ccc?key=blah-blah-blah-blah#gid=1'
result = re.search(r'^http[s]*:\/\/[\w\.]*', url).group()
print(result)

#result
'https://docs.google.com'

回答 12

获取域名/主机名和来源*

url = '/programming/9626535/get-protocol-host-name-from-url'
hostname = url.split('/')[2] # stackoverflow.com
origin = '/'.join(url.split('/')[:3]) # https://stackoverflow.com

* Origin用于XMLHttpRequest标题

点击查看英文原文

to get domain/hostname and Origin*

url = 'https://stackoverflow.com/questions/9626535/get-protocol-host-name-from-url'
hostname = url.split('/')[2] # stackoverflow.com
origin = '/'.join(url.split('/')[:3]) # https://stackoverflow.com

*Origin is used in XMLHttpRequest headers

回答 13

您可以简单地使用带有相对根“ /”的urljoin作为第二个参数：

try:
    from urlparse import urljoin  # Python2
except ImportError:
    from urllib.parse import urljoin  # Python3


url = '/programming/9626535/get-protocol-host-name-from-url'

root_url = urljoin(url, '/')
点击查看英文原文

You can simply use urljoin with relative root '/' as second argument:

import urllib.parse


url = 'https://stackoverflow.com/questions/9626535/get-protocol-host-name-from-url'
root_url = urllib.parse.urljoin(url, '/')
print(root_url)

回答 14

如果它包含的斜线少于3个，则说明您已经得到了，如果不是，那么我们可以发现它之间的出现：

import re

link = http://forum.unisoftdev.com/something

slash_count = len(re.findall("/", link))
print slash_count # output: 3

if slash_count > 2:
   regex = r'\:\/\/(.*?)\/'
   pattern  = re.compile(regex)
   path = re.findall(pattern, url)

   print path
点击查看英文原文

If it contains less than 3 slashes thus you've it got and if not then we can find the occurrence between it:

import re

link = http://forum.unisoftdev.com/something

slash_count = len(re.findall("/", link))
print slash_count # output: 3

if slash_count > 2:
   regex = r'\:\/\/(.*?)\/'
   pattern  = re.compile(regex)
   path = re.findall(pattern, url)

   print path
Django(254)Python(4302)
本文由 Python 实用宝典 作者：Python实用宝典 发表，其版权均为 Python 实用宝典 所有，文章内容系作者个人观点，不代表 Python 实用宝典 对观点赞同或支持。如需转载，请注明文章来源。
♥已赞15

相关文章

抱歉，评论已关闭！