GUIDE · PARQUET + PYTHON · CATALOG VERIFIED 2026-11-11

Working with Parquet in Python for options backtesting

Options Data (optionsdata.shop) delivers NIFTY and SENSEX option chains as Parquet, one file per trading day per expiry, covering NIFTY 1-minute JAN 2023 → NOV 2026. Parquet is columnar and typed, so pandas, PyArrow and DuckDB read only the columns and rows you request. This guide shows how to load one file, query many days at once, handle the two column layouts, and backtest a simple NIFTY straddle without look-ahead. Data: NIFTY option chain historical data.

NIFTY option chain data NIFTY options, 3 years Get free sample by email File schema

Get the free sample by email → check the columns

Dates on this page are read from the live file catalog, not typed by hand. Full matrix: /coverage.

Read less: columns and filters

Install pandas, pyarrow and duckdb. Reading a whole file is fine for one day; across years, read only what the test uses. PyArrow pushes filters down so unneeded row groups are skipped.

import pandas as pd
cols = ["datetime", "strike_price", "right", "close", "volume", "open_interest"]
df = pd.read_parquet("NIFTY_OPT_1min_2026-09-10_exp2026-09-15.parquet",
                     columns=cols,
                     filters=[("strike_price", ">=", 24500), ("strike_price", "<=", 25500)])

Query many days at once with DuckDB

DuckDB reads a folder of Parquet files as one table without loading it into memory first. union_by_name lines up files from both column layouts, and filename keeps the source file so you can recover the trading day and expiry.

import duckdb
q = r"""
SELECT regexp_extract(filename, '_exp(\d{4}-\d{2}-\d{2})', 1) AS expiry, *
FROM read_parquet('data/NIFTY_OPT_1min_*.parquet', union_by_name=true, filename=true)
WHERE strftime(CAST(datetime AS TIMESTAMP), '%H:%M') = '09:20'  -- layout B files: "timestamp"
"""
entry = duckdb.sql(q).df()

Two column layouts, one loader

The archive uses two column layouts (listed on the file schema page). Layout A, used by the sample and recent files, names the columns datetime, strike_price, right, expiry_date and open_interest, with Call/Put and expiries like 15-SEP-2026. Layout B uses timestamp, strike, option_type, expiry and oi, with CE/PE and expiries like 2026-09-15.

Rename straight after reading so every later line of your code sees one set of names, and parse the expiry with an explicit format for each layout instead of letting pandas guess day and month order:

import pandas as pd
df = pd.read_parquet("NIFTY_OPT_1min_2026-09-10_exp2026-09-15.parquet")
# older files use layout B names; map them to the sample names
df = df.rename(columns={"timestamp": "datetime", "strike": "strike_price",
                        "option_type": "right", "expiry": "expiry_date",
                        "oi": "open_interest"})
if "right" in df:
    df["right"] = df["right"].replace({"CE": "Call", "PE": "Put"})
def parse_expiry(s):
    s = s.astype(str)
    a = pd.to_datetime(s, format="%d-%b-%Y", errors="coerce")
    b = pd.to_datetime(s, format="%Y-%m-%d", errors="coerce")
    return a.fillna(b)
df["expiry_dt"] = parse_expiry(df["expiry_date"])
df["datetime"] = pd.to_datetime(df["datetime"])  # IST wall-clock, bar start

A straddle backtest without look-ahead

Sell the at-the-money straddle of the nearest expiry at 09:20 and buy it back at 15:15. The ATM strike is chosen from the 09:20 bars only: the strike where call and put closes are closest. Prices are the last traded close at or before each time, because a strike may not trade in the exact minute.

import glob, re
import pandas as pd

# nearest expiry per trading day, from the file names
near = {}
for p in glob.glob("data/NIFTY_OPT_1min_*_exp*.parquet"):
    d, e = re.search(r"_(\d{4}-\d{2}-\d{2})_exp(\d{4}-\d{2}-\d{2})", p).groups()
    if d not in near or e < near[d][0]:
        near[d] = (e, p)

def px(df, t):
    s = df[df["datetime"] <= t].groupby(["strike_price", "right"])["close"].last()
    return s.unstack("right")

out = []
for d, (e, p) in sorted(near.items()):
    df = pd.read_parquet(p)
    # older files use layout B names; map them to the sample names
    df = df.rename(columns={"timestamp": "datetime", "strike": "strike_price",
                            "option_type": "right", "expiry": "expiry_date",
                            "oi": "open_interest"})
    if "right" in df:
        df["right"] = df["right"].replace({"CE": "Call", "PE": "Put"})
    df["datetime"] = pd.to_datetime(df["datetime"])
    a = px(df, pd.Timestamp(f"{d} 09:20:00")).dropna()
    if a.empty:
        continue
    k = (a["Call"] - a["Put"]).abs().idxmin()
    b = px(df, pd.Timestamp(f"{d} 15:15:00"))
    credit = a.loc[k, "Call"] + a.loc[k, "Put"]
    debit = b.loc[k, "Call"] + b.loc[k, "Put"]
    out.append({"day": d, "expiry": e, "strike": k, "pnl_points": credit - debit})

res = pd.DataFrame(out)
print(res["pnl_points"].describe())

Habits that keep results honest

Add costs and slippage to every fill: the files hold traded prices, so assume you pay away from the close. Never pick strikes or filters using values from later in the day. Keep contract identity (strike, expiry, call/put) on every trade so expired contracts are never spliced into a rolling series.

Write intermediate results back to Parquet (df.to_parquet("results.parquet")) instead of CSV, so types survive between runs. For a first look at one file see the pandas loading guide.

Exactly which fields are in the files

Options files

Column (layout A · layout B)	Type	Meaning
datetime · timestamp	timestamp	Bar start time, IST wall-clock (e.g. 2026-09-10 09:15:00)
stock_code · —	string	Underlying code: NIFTY, or BSESEN for SENSEX
exchange_code · —	string	NFO (NSE F&O) or BFO (BSE F&O)
product_type · —	string	Options
expiry_date · expiry	string	Contract expiry (A: 15-SEP-2026, B: 2026-09-15)
strike_price · strike	number	Strike price
right · option_type	string	A: Call / Put, B: CE / PE
open / high / low / close	float	Traded premium OHLC within the bar
volume	int	Volume traded in the bar
open_interest · oi	int	Open interest at that bar (SENSEX has gaps where it is 0 — see /free/schema)

Each row is one contract in one bar: traded OHLC, volume and open interest, with strike, expiry and call/put on every row.

Format: Parquet. Validate on the free sample and the schema page before buying.

FAQ

Why use Parquet instead of CSV for options backtesting?

Parquet stores typed columns compressed, so files are smaller and you can read only the columns and strikes you need. CSV must be parsed in full and loses types.

How do I read many option Parquet files at once?

Use DuckDB: read_parquet with a glob pattern, union_by_name=true for mixed layouts and filename=true to keep the trading day and expiry.

Can I convert the Parquet files to CSV?

Yes, with one line of pandas: pd.read_parquet(path).to_csv(out, index=False). Every file can also be downloaded as CSV.

How do I avoid look-ahead bias in an options backtest?

Choose strikes and signals only from bars at or before the decision time, use the last traded close at or before each time, and add costs to every fill.

Which data does this code expect?

NIFTY option chain files at 1-minute from the NIFTY options historical data page, or the free sample, which uses the same layout.

Get the data

Free sample · Schema · Ask the AI for your data

NIFTY option chain data NIFTY options, 3 years Get free sample by email