import Python, Finance, Scientific Computing - Latest Comments

Re: Even easier frequency tables in pandas 0.7.0

Wes McKinney — Tue, 07 May 2013 16:51:03 -0000

I don't think it does at the moment but could be enabled to do so

Re: Even easier frequency tables in pandas 0.7.0

mittenchops — Tue, 07 May 2013 12:17:40 -0000

Does this support crosstabs of array objects? Say df['b'] had contained multiple elements.

Re: I’m moving to San Francisco. And hiring

Rooz — Thu, 02 May 2013 15:32:50 -0000

Wes, would love to get you to speak at our Big Data Science meetup in San Francisco some time. Please message me if your interested.
-Rooz

Re: NumPy indexing peculiarities

Wes McKinney — Tue, 30 Apr 2013 18:05:15 -0000

I haven't done any profiling with recent builds of NumPy-- I know there's still more work to be done.

Re: NumPy indexing peculiarities

dbv — Sun, 28 Apr 2013 05:16:43 -0000

Did this ever get resolved in newer releases of Numpy?

Re: Why I’m not on the Julia bandwagon (yet)

MySchizoBuddy — Wed, 17 Apr 2013 08:29:13 -0000

ur title says "not yet" what's the minimum improvement that you think julia should have before you start reconsidering.

Re: I’m moving to San Francisco. And hiring

Aidan Lorcan — Wed, 10 Apr 2013 06:18:38 -0000

I assume the node.js is for concurrency and performance reasons(two of pythons natural weak points) to stream the client datasets to the backend for analysis in cythonised python.

Anyway best of luck Wes, I wont be joining you but I may be using your product.The GoG work in particular looks very interesting.

Re: I’m moving to San Francisco. And hiring

MySchizoBuddy — Mon, 01 Apr 2013 16:19:46 -0000

newbie question. you are asking for both python and node.js experience. I didn't know they can work together. I thought it it is either node.js or python on the server. Can you talk about this requirement and why a python developer should learn node.js?

Re: I’m moving to San Francisco. And hiring

John Marino — Wed, 27 Mar 2013 11:19:17 -0000

Ditto on the wishing-well.

Re: I’m moving to San Francisco. And hiring

Francesc — Sat, 23 Mar 2013 19:59:28 -0000

Well deserved. Good luck in your new venture!

Re: I’m moving to San Francisco. And hiring

Miki — Sat, 23 Mar 2013 11:35:39 -0000

Good luck!

Re: I’m moving to San Francisco. And hiring

h6o6 — Sat, 23 Mar 2013 00:28:54 -0000

Glad to hear things are going well, still loving pandas daily, so hopefully development won't drop too much there... guess I'll have to get off my ass and contribute :) Best of luck!

Re: I’m moving to San Francisco. And hiring

Jacob Frelinger — Fri, 22 Mar 2013 20:54:24 -0000

Awesome to hear things are going so well for you and the start-up

Re: Update on upcoming pandas v0.10, new file parser, other performance wins

durden20 — Fri, 08 Mar 2013 11:42:24 -0000

Oh ok, that makes a lot of sense. I did look into read_csv but not loadtxt. Thanks for the feedback!

Re: Update on upcoming pandas v0.10, new file parser, other performance wins

Wes McKinney — Fri, 08 Mar 2013 10:53:30 -0000

Short answer: file tokenization and type inference is being handled at the lowest level possible in C/Cython. If you look at the impl of numpy.loadtxt you'll see a lot of Python.

Re: Update on upcoming pandas v0.10, new file parser, other performance wins

durden20 — Fri, 08 Mar 2013 10:49:39 -0000

Guess the answer is pretty much already on your blog! For others to reference:

- Fast new file parser added: http://wesmckinney.com/blog/?m...

- Speeding up file parsing with Cython: http://wesmckinney.com/blog/?p...

Re: Update on upcoming pandas v0.10, new file parser, other performance wins

durden20 — Fri, 08 Mar 2013 10:37:33 -0000

It's great to see read_csv get such a big improvement. I'm curious as to why or the short version of why read_csv is so much faster than numpy.loadxt()?

For example, I'm comparing numbers on a relatively small space-separated file (about 208K) with 26 columns and 1000 rows. Each item in the file is a random floating point number that I generated with numpy.rand.

>>> %timeit f = open('test.out', 'r');f.readline();np.loadtxt(f, unpack=True)
10 loops, best of 3: 26.9 ms per loop

>>> %timeit pd.read_csv('test.out', delim_whitespace=True)
100 loops, best of 3: 5.99 ms per loop

Re: Whirlwind tour of pandas in 10 minutes

Michael Paone — Tue, 12 Feb 2013 17:35:40 -0000

Nevermind, I figured out it's IPython. Great video!

Re: Whirlwind tour of pandas in 10 minutes

Michael Paone — Tue, 12 Feb 2013 17:33:06 -0000

What IDE are you using in the video?

Re: Whirlwind tour of pandas in 10 minutes

dartdog — Tue, 12 Feb 2013 07:54:23 -0000

always helpful..

Re: Whirlwind tour of pandas in 10 minutes

Will C — Mon, 11 Feb 2013 17:38:11 -0000

Wes, Great overview. Keep them coming!

Re: Update on upcoming pandas v0.10, new file parser, other performance wins

Wes McKinney — Fri, 08 Feb 2013 12:45:39 -0000

Can you post this on GitHub please? https://github.com/pydata/pand...

Re: Update on upcoming pandas v0.10, new file parser, other performance wins

Feng — Fri, 08 Feb 2013 00:09:31 -0000

I tried this on my Mac with 8GB memory, but I got this:

Python(12618,0xacb01a28) malloc: *** mmap(size=24051712) failed (error code=12)
*** error: can't allocate region
*** set a breakpoint in malloc_error_break to debug

what is possible reason for this? thanks for your help

Re: A Roadmap for Rich Scientific Data Structures in Python

sfermigier — Fri, 01 Feb 2013 11:42:39 -0000

Don't bother with this bozo, Wes. What you've already achieved over the last 18 months is just fantastic. Keep up the good work !

Re: A Roadmap for Rich Scientific Data Structures in Python

Wes McKinney — Thu, 31 Jan 2013 15:23:44 -0000

So you're saying it's wrong to strive to improve the status quo and thus I shouldn't be taken seriously (your initial comment was "people who use phrases like this "most powerful data analysis stack to ever exist" shouldn't be taken seriously") ? I think if you look at what I've done (hacked, shipped, written) in the 18 months since writing this blog post you'll see that I've more than delivered on my goal to make Python a compelling choice for data analysis.