Python: reading two columns of numerical data with an arbitrary number of header & blank lines

Here is a method.

The input data may be either space, tab, or comma delimited.

##########################################################################

import os
import re

from numpy import array,zeros,concatenate,linspace,std
from sys import stdin

import matplotlib.pyplot as plt

########################################################################

def start():
print(” “)
print(“Entpath =stdin.readline()
file_path = input_file_path.rstrip(‘\n’)
if os.path.exists(file_path):
print “This file exists”
print ” ”
infile = open(file_path,”rb”)
lines = infile.readlines()
infile.close()
if not os.path.exists(file_path):
print “This file doesn’t exist”
start()
return lines

########################################################################

def lines_two_columns():
#
# http://flockhart.virtualave.net/RBIF0100/regexp.html
#
lines=start()
a = []
b = []
num=0
for line in lines:
#
if re.search(r”(\d+)”, line): # matches a digit
iflag=0
else:
iflag=1 # did not find digit
#
if re.search(r”#”, line):
iflag=1
#
if iflag==0:
line=line.lower()
if re.search(r”([a-d])([f-z])”, line): # ignore header lines
iflag=1
else:
line = line.replace(“,”,” “)
col1,col2=line.split()
a.append(float(col1))
b.append(float(col2))
num=num+1
return a,b,num

##########################################################################

a,b,num =lines_two_columns()

* * *
Tom Irvine

Cython

Introduction

Cython is a compiler which compiles Python-like code files to C code. The resulting functions can then be imported into other Python scripts.

This method can be used to increase the execution speed of a Python script, particularly if the script uses for-loops.

The Python-like *.pyx script which provides the function must have have static C data types in order for the main Python script to gain full speed performance.

Installation

Cython can be installed on a Ubuntu system using the “Ubuntu Software Center.”

It is also bundled with the Enthought Python Distribution [EPD] and with Sage Notebook.

Steps

Using Cython consists of these steps:

1. Write a .pyx source file for the function

2. Write a setup.py file using the disutils function

3. Compile the source code in command-line mode using: python setup.py build_ext –inplace

4. Import the function into the main Python script and then run it
There are a few alternative compiler methods, but disutils is the most reliable from the author’s experience.

Examples

Cython: Runge-Kutta

Cython: Fourier transform

Tom Irvine