问题:熊猫:在Excel文件中查找工作表列表
新版本的Pandas使用以下界面加载Excel文件:
read_excel('path_to_file.xls', 'Sheet1', index_col=None, na_values=['NA'])
但是,如果我不知道可用的图纸怎么办?
例如,我正在使用以下工作表的excel文件
数据1,数据2 …,数据N,foo,bar
但我不知道N
先验。
有什么方法可以从Pandas的excel文档中获取工作表列表吗?
The new version of Pandas uses the following interface to load Excel files:
read_excel('path_to_file.xls', 'Sheet1', index_col=None, na_values=['NA'])
but what if I don’t know the sheets that are available?
For example, I am working with excel files that the following sheets
Data 1, Data 2 …, Data N, foo, bar
but I don’t know N
a priori.
Is there any way to get the list of sheets from an excel document in Pandas?
回答 0
您仍然可以使用ExcelFile类(和sheet_names
属性):
xl = pd.ExcelFile('foo.xls')
xl.sheet_names # see all sheet names
xl.parse(sheet_name) # read a specific sheet to DataFrame
有关更多选项,请参阅文档以进行解析 …
You can still use the ExcelFile class (and the sheet_names
attribute):
xl = pd.ExcelFile('foo.xls')
xl.sheet_names # see all sheet names
xl.parse(sheet_name) # read a specific sheet to DataFrame
see docs for parse for more options…
回答 1
您应该将第二个参数(工作表名称)明确指定为“无”。像这样:
df = pandas.read_excel("/yourPath/FileName.xlsx", None);
“ df”都是作为DataFrames字典的工作表,您可以通过运行以下命令进行验证:
df.keys()
结果是这样的:
[u'201610', u'201601', u'201701', u'201702', u'201703', u'201704', u'201705', u'201706', u'201612', u'fund', u'201603', u'201602', u'201605', u'201607', u'201606', u'201608', u'201512', u'201611', u'201604']
请参阅pandas doc了解更多详细信息: https //pandas.pydata.org/pandas-docs/stable/generation/pandas.read_excel.html
You should explicitly specify the second parameter (sheetname) as None. like this:
df = pandas.read_excel("/yourPath/FileName.xlsx", None);
“df” are all sheets as a dictionary of DataFrames, you can verify it by run this:
df.keys()
result like this:
[u'201610', u'201601', u'201701', u'201702', u'201703', u'201704', u'201705', u'201706', u'201612', u'fund', u'201603', u'201602', u'201605', u'201607', u'201606', u'201608', u'201512', u'201611', u'201604']
please refer pandas doc for more details: https://pandas.pydata.org/pandas-docs/stable/generated/pandas.read_excel.html
回答 2
这是我发现最快的方法,灵感来自@divingTobi的答案。所有基于xlrd,openpyxl或pandas的答案对我来说都很慢,因为它们都首先加载整个文件。
from zipfile import ZipFile
from bs4 import BeautifulSoup # you also need to install "lxml" for the XML parser
with ZipFile(file) as zipped_file:
summary = zipped_file.open(r'xl/workbook.xml').read()
soup = BeautifulSoup(summary, "xml")
sheets = [sheet.get("name") for sheet in soup.find_all("sheet")]
This is the fastest way I have found, inspired by @divingTobi’s answer. All The answers based on xlrd, openpyxl or pandas are slow for me, as they all load the whole file first.
from zipfile import ZipFile
from bs4 import BeautifulSoup # you also need to install "lxml" for the XML parser
with ZipFile(file) as zipped_file:
summary = zipped_file.open(r'xl/workbook.xml').read()
soup = BeautifulSoup(summary, "xml")
sheets = [sheet.get("name") for sheet in soup.find_all("sheet")]
回答 3
以@dhwanil_shah的答案为基础,您不需要提取整个文件。有了zf.open
它,可以直接从一个压缩文件中读取。
import xml.etree.ElementTree as ET
import zipfile
def xlsxSheets(f):
zf = zipfile.ZipFile(f)
f = zf.open(r'xl/workbook.xml')
l = f.readline()
l = f.readline()
root = ET.fromstring(l)
sheets=[]
for c in root.findall('{http://schemas.openxmlformats.org/spreadsheetml/2006/main}sheets/*'):
sheets.append(c.attrib['name'])
return sheets
连续两个 readline
s很难看,但内容仅在文本的第二行中。无需解析整个文件。
该解决方案似乎比该read_excel
版本要快得多,而且很有可能比完整提取版本还快。
Building on @dhwanil_shah ‘s answer, you do not need to extract the whole file. With zf.open
it is possible to read from a zipped file directly.
import xml.etree.ElementTree as ET
import zipfile
def xlsxSheets(f):
zf = zipfile.ZipFile(f)
f = zf.open(r'xl/workbook.xml')
l = f.readline()
l = f.readline()
root = ET.fromstring(l)
sheets=[]
for c in root.findall('{http://schemas.openxmlformats.org/spreadsheetml/2006/main}sheets/*'):
sheets.append(c.attrib['name'])
return sheets
The two consecutive readline
s are ugly, but the content is only in the second line of the text. No need to parse the whole file.
This solution seems to be much faster than the read_excel
version, and most likely also faster than the full extract version.
回答 4
我已经尝试过xlrd,pandas,openpyxl和其他类似的库,并且随着读取整个文件时文件大小的增加,它们似乎都花费了指数时间。上面提到的其他使用’on_demand’的解决方案对我不起作用。如果只想最初获取工作表名称,则以下功能适用于xlsx文件。
def get_sheet_details(file_path):
sheets = []
file_name = os.path.splitext(os.path.split(file_path)[-1])[0]
# Make a temporary directory with the file name
directory_to_extract_to = os.path.join(settings.MEDIA_ROOT, file_name)
os.mkdir(directory_to_extract_to)
# Extract the xlsx file as it is just a zip file
zip_ref = zipfile.ZipFile(file_path, 'r')
zip_ref.extractall(directory_to_extract_to)
zip_ref.close()
# Open the workbook.xml which is very light and only has meta data, get sheets from it
path_to_workbook = os.path.join(directory_to_extract_to, 'xl', 'workbook.xml')
with open(path_to_workbook, 'r') as f:
xml = f.read()
dictionary = xmltodict.parse(xml)
for sheet in dictionary['workbook']['sheets']['sheet']:
sheet_details = {
'id': sheet['@sheetId'],
'name': sheet['@name']
}
sheets.append(sheet_details)
# Delete the extracted files directory
shutil.rmtree(directory_to_extract_to)
return sheets
由于所有xlsx基本上都是压缩文件,因此我们提取基本的xml数据并直接从工作簿中读取工作表名称,与库函数相比,此过程只需花费一秒钟的时间。
基准测试:(在具有4张纸的
6mb xlsx文件上)Pandas,xlrd: 12秒
openpyxl: 24秒
建议的方法: 0.4秒
由于我的要求只是读取工作表名称,因此读取整个时间不必要的开销困扰着我,所以我改用了这种方法。
I have tried xlrd, pandas, openpyxl and other such libraries and all of them seem to take exponential time as the file size increase as it reads the entire file. The other solutions mentioned above where they used ‘on_demand’ did not work for me. If you just want to get the sheet names initially, the following function works for xlsx files.
def get_sheet_details(file_path):
sheets = []
file_name = os.path.splitext(os.path.split(file_path)[-1])[0]
# Make a temporary directory with the file name
directory_to_extract_to = os.path.join(settings.MEDIA_ROOT, file_name)
os.mkdir(directory_to_extract_to)
# Extract the xlsx file as it is just a zip file
zip_ref = zipfile.ZipFile(file_path, 'r')
zip_ref.extractall(directory_to_extract_to)
zip_ref.close()
# Open the workbook.xml which is very light and only has meta data, get sheets from it
path_to_workbook = os.path.join(directory_to_extract_to, 'xl', 'workbook.xml')
with open(path_to_workbook, 'r') as f:
xml = f.read()
dictionary = xmltodict.parse(xml)
for sheet in dictionary['workbook']['sheets']['sheet']:
sheet_details = {
'id': sheet['@sheetId'],
'name': sheet['@name']
}
sheets.append(sheet_details)
# Delete the extracted files directory
shutil.rmtree(directory_to_extract_to)
return sheets
Since all xlsx are basically zipped files, we extract the underlying xml data and read sheet names from the workbook directly which takes a fraction of a second as compared to the library functions.
Benchmarking: (On a 6mb xlsx file with 4 sheets)
Pandas, xlrd: 12 seconds
openpyxl: 24 seconds
Proposed method: 0.4 seconds
Since my requirement was just reading the sheet names, the unnecessary overhead of reading the entire time was bugging me so I took this route instead.
回答 5
from openpyxl import load_workbook
sheets = load_workbook(excel_file, read_only=True).sheetnames
对于我正在使用的5MB Excel文件,load_workbook
没有read_only
标记花费了8.24秒。带有read_only
标志,仅花费了39.6 ms。如果您仍然想使用Excel库而不是使用xml解决方案,那将比解析整个文件的方法快得多。
from openpyxl import load_workbook
sheets = load_workbook(excel_file, read_only=True).sheetnames
For a 5MB Excel file I’m working with, load_workbook
without the read_only
flag took 8.24s. With the read_only
flag it only took 39.6 ms. If you still want to use an Excel library and not drop to an xml solution, that’s much faster than the methods that parse the whole file.