December 2015 Archives

2015-12-21 21:54:32

View From Mount Stupid

I was dealing with family issues most of yesterday and today, but I am still plugging away at this.

I am now in my third day of the StockFighter challenge. two days ago you might recall, I managed to get the automated web browser httpie installed but it seems to have some problems with SSL.  I will spend a few cycles resolving those issues, but for the time being I want to focus on the other method mentioned in the API guide using the ruby language. Ruby is an interesting language. It was developed in Japan and the main developers speak Japanese.  Some of the discussion on the forums is exclusively in Japanese.  This would not bother me if the language was in a stable condition, like for instance C language, but it is not.  Ruby is in active development 20 years after it was released.  Some people call this "remaining relevant". I call it gratuitous fluff.  So, back to the challenge. The ruby program to handle the first trade is here:
https://starfighter.readme.io/docs/getting-started

I decided to try it. So here goes:


[starfighter@sl7-vm stockfighter]$ ruby first-trade.rb
/usr/share/rubygems/rubygems/core_ext/kernel_require.rb:55:in `require': cannot load such file -- httparty (LoadError)
        from /usr/share/rubygems/rubygems/core_ext/kernel_require.rb:55:in `require'
        from first-trade.rb:2:in `<main>'


Let's read the docs ...

[starfighter@sl7-vm stockfighter]$ sudo gem install httparty
[sudo] password for starfighter:

Fetching: json-1.8.3.gem (100%)
Building native extensions.  This could take a while...
ERROR:  Error installing httparty:
        ERROR: Failed to build gem native extension.

    /usr/bin/ruby extconf.rb
mkmf.rb can't find header files for ruby at /usr/share/include/ruby.h


Poo!


[root@sl7-vm ~]# yum install ruby-devel rubygems-devel rubygem-net-http-persistent ruby-doc rubygem-json

[snip]

Installed:
  ruby-devel.x86_64 0:2.0.0.598-25.el7_1                      ruby-doc.noarch 0:2.0.0.598-25.el7_1
  rubygem-net-http-persistent.noarch 0:2.8-5.el7              rubygems-devel.noarch 0:2.0.14-25.el7_1

Complete!
[root@sl7-vm ~]# gem install httparty
Fetching: json-1.8.3.gem (100%)
Building native extensions.  This could take a while...
ERROR:  Error installing httparty:
        ERROR: Failed to build gem native extension.

    /usr/bin/ruby extconf.rb
creating Makefile

make "DESTDIR="
gcc -I. -I/usr/include -I/usr/include/ruby/backward -I/usr/include -I. -DJSON_GENERATOR    -fPIC -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -mtune=generic -fPIC -m64 -o generator.o -c generator.c
make: gcc: Command not found
make: *** [generator.o] Error 127


Double poo! After 20 years don't you think this POS( Piece of Software ) should be self hosting?


[root@sl7-vm ~]# yum install gcc
Loaded plugins: langpacks
Resolving Dependencies

Installed:
  gcc.x86_64 0:4.8.3-9.el7

Dependency Installed:
  cpp.x86_64 0:4.8.3-9.el7                    glibc-devel.x86_64 0:2.17-78.el7 glibc-headers.x86_64 0:2.17-78.el7
  kernel-headers.x86_64 0:3.10.0-229.20.1.el7 libmpc.x86_64 0:1.0.1-3.el7      mpfr.x86_64 0:3.1.1-4.el7

Complete!
[root@sl7-vm ~]# gem install httparty
Building native extensions.  This could take a while...
Successfully installed json-1.8.3
Fetching: multi_xml-0.5.5.gem (100%)
Successfully installed multi_xml-0.5.5
Fetching: httparty-0.13.7.gem (100%)
When you HTTParty, you must party hard!
Successfully installed httparty-0.13.7
Parsing documentation for json-1.8.3
Installing ri documentation for json-1.8.3
Parsing documentation for multi_xml-0.5.5
Installing ri documentation for multi_xml-0.5.5
Parsing documentation for httparty-0.13.7
Installing ri documentation for httparty-0.13.7


Whee!!!!  Now we can party!

[starfighter@sl7-vm stockfighter]$ ruby first-trade.rb
{
  "ok": true,
  "symbol": "RKI",
  "venue": "YLMEX",
  "direction": "buy",
  "originalQty": 25,
  "qty": 25,
  "price": 8700,
  "orderType": "limit",
  "id": 5157,
  "account": "<redacted>",
  "ts": "2015-12-20T18:52:29.266964823Z",
  "fills": [],
  "totalFilled": 0,
  "open": true
}


So I'm making progress.  I'm wondering how many people will choose to fight this octopus to get to the stage of being able to launch an order.  I feel like I am junking up my machine and introducing security holes so I can run some idiosyncratic garbage that will be replaced in a few days.  That is not a exaggeration. Ruby 2.2.4 was released Dec. 16, 2015.  You can bet that most of those "gems" will need to be re-written to cope.  The last version of C was released in 2011.  If you where a C programmer you might not have noticed.  The C language is standardized ( via ANSI in 1989 or ISO in 1990 ).  Your programs would continue to work without rewrites.

I read an article yesterday about Mount Stupid:
mount stupid

The premise is that people that know nothing about a subject will broadcast their ignorance to the world.  So I am just climbing the front side of Mount Stupid right now.  I expect the decent to be a long trip so you can expect to hear from me for a long time.


Posted by StarFighter | Permanent link

2015-12-20 01:53:57

First Day and First Level

If you are making a game about programming, it seems to me that you should make it as accessible as possible. This is not a rant. I know next to nothing about programming and if someone introduces me to a new tool I don't reject it out of hand. However the very first level introduces me to the StockFighter API with this text:

Shell

# Let's try something different this time.  Sure, you could interact with the API
# in an actual programming language, but let me introduce you to httpie, a very
# cool tool.

# httpie is like curl, except much nicer to use.  See: https://github.com/jkbrzt/httpie

# You can install it via: 
# brew install httpie
# apt-get install httpie
# yum install httpie

# We'll start by creating a file representing our JSON request, like:

$ nano /tmp/order.json

# Copy/paste in:

{
  "account": "MST92145671",
  "venue": "LOBHEX",
  "stock": "LPEI",
  "qty": 100,
  "direction": "buy",
  "orderType": "market" 
}

# Got that ready?  Great, here's httpie magic:

$ http POST https://api.stockfighter.io/ob/api/venues/LOBHEX/stocks/LPEI/orders 'X-Starfighter-Authorization:copy-and-paste-your-API-key-here' < /tmp/order.json

# Now isn't that so much nicer than curl?
# No unnecessary headers (the Accept: application/json is implict.)
# You can inspect the requests quickly, with syntax highlighting.

# This is frequently useful when you're exploring a novel API.  Try it out!
# Most developers, after they get a handle on the endpoints, will want to
# write a more robust program in their language of choice, but httpie is
# a good choice for interactive programming when you want to figure out
# exactly what is happening when e.g. a library isn't working as expected.


Cool Right? I just put together a new virtual machine and got this blog software working. Now I want to try this API thingy so lets Fire up that httpie and see what happens!

[root@sl7-vm ~]# yum install httpie
Loaded plugins: langpacks
No package httpie available.
Error: Nothing to do

So I'm not easily dissuaded. I check out the web site here.

It says you can use something called "pip" if your favorite yum repo doesn't happen to have hippie in it. Let's try that:

[root@sl7-vm ~]# pip
-bash: pip: command not found

Hummm....

[root@sl7-vm ~]# easy_install pip
Searching for pip
Reading https://pypi.python.org/simple/pip/
Best match: pip 7.1.2
Downloading https://pypi.python.org/packages/source/p/pip/pip-7.1.2.tar.gz#md5=3823d2343d9f3aaab21cf9c917710196
Processing pip-7.1.2.tar.gz
Writing /tmp/easy_install-jC2noC/pip-7.1.2/setup.cfg
Running pip-7.1.2/setup.py -q bdist_egg --dist-dir /tmp/easy_install-jC2noC/pip-7.1.2/egg-dist-tmp-tBPFxY
warning: no previously-included files found matching '.coveragerc'
warning: no previously-included files found matching '.mailmap'
warning: no previously-included files found matching '.travis.yml'
warning: no previously-included files found matching 'pip/_vendor/Makefile'
warning: no previously-included files found matching 'tox.ini'
warning: no previously-included files found matching 'dev-requirements.txt'
no previously-included directories found matching '.travis'
no previously-included directories found matching 'docs/_build'
no previously-included directories found matching 'contrib'
no previously-included directories found matching 'tasks'
no previously-included directories found matching 'tests'
Adding pip 7.1.2 to easy-install.pth file
Installing pip script to /usr/bin
Installing pip2.7 script to /usr/bin
Installing pip2 script to /usr/bin

Installed /usr/lib/python2.7/site-packages/pip-7.1.2-py2.7.egg
Processing dependencies for pip
Finished processing dependencies for pip

[root@sl7-vm ~]# pip install --upgrade pip setuptools
/usr/lib/python2.7/site-packages/pip-7.1.2-py2.7.egg/pip/_vendor/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Requirement already up-to-date: pip in /usr/lib/python2.7/site-packages/pip-7.1.2-py2.7.egg
Collecting setuptools
Downloading setuptools-19.1.1-py2.py3-none-any.whl (463kB)
100% |████████████████████████████████| 466kB 330kB/s
Installing collected packages: setuptools
Found existing installation: setuptools 0.9.8
Uninstalling setuptools-0.9.8:
Successfully uninstalled setuptools-0.9.8
Successfully installed setuptools-19.1.1
/usr/lib/python2.7/site-packages/pip-7.1.2-py2.7.egg/pip/_vendor/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning

[root@sl7-vm ~]# pip install --upgrade httpie
Collecting httpie
/usr/lib/python2.7/site-packages/pip-7.1.2-py2.7.egg/pip/_vendor/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Downloading httpie-0.9.2-py2.py3-none-any.whl (66kB)
100% |████████████████████████████████| 69kB 556kB/s
Collecting requests>=2.3.0 (from httpie)
Downloading requests-2.9.0-py2.py3-none-any.whl (500kB)
100% |████████████████████████████████| 503kB 235kB/s
Collecting Pygments>=1.5 (from httpie)
Downloading Pygments-2.0.2-py2-none-any.whl (672kB)
100% |████████████████████████████████| 675kB 464kB/s
Installing collected packages: requests, Pygments, httpie
Successfully installed Pygments-2.0.2 httpie-0.9.2 requests-2.9.0
[root@sl7-vm ~]#


So what happened there? It looks to me like I just installed a package outside of the rpm database. In general I try to avoid doing that. It can lead to a situation known as "Dependency Hell". This was why the yum system was introduced. The intention was to avoid having to install packages from source. You will note that the pip package and the httpie package spray files all over the place. In addition this new POS ( Piece of Software) has introduced a security vulnerability into my relatively clean box. It is a pretty minor one, but it is not simple to fix. I am running Scientific Linux, a RedHat derivative. This is important because RedHat uses python for all of it's system software. Leaving aside for the moment advisability of writing crucial system software in an interpreted language, lets look at this security issue . The message I get when I try to install httpie says this:

InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.

The linked page tells me that my version of python needs a tweak (or three) to handle HTTPS requests. The recommended fix is to "upgrade" python to version 2.7.9 ( am running Python 2.7.5 ). I downloaded this OS today. It still has that new car smell. I will not "upgrade" python.


Long story short, This is a crappy tool. Why didn't you show an example using the tools that come with the OS? In my case that includes wget, cURL and ... what else? elinks maybe? There is another example on the page that uses ruby. I will write about that next time.

-StarFighter



Posted by StarFighter | Permanent link

2015-12-19 15:21:10

Stockfighter blog by StarFighter

This is my blog about the StockFighter on-line programming challenge. Stockfighter is an online programming "game" that requires you to write an automated trading system ( or several of them ).  This will be a difficult assignment, so I am writing this blog to serve as a quick reference and to motivate myself to complete the challenge.  The blog will almost certainly contain spoilers so don't read it if you want to find the answers yourself.  My technique will evolve as I go along so I don't know what language I will write in or what systems I will use.  The only thing I am sure of is that I do not wish to use any hints or assistance.  I have almost no experience in programming or stock markets so this will be a real education.  I expect that it will take me a long time to finish.  I hope that I will not be the last one to finish the challenge, but if I am the last one, then I will become "The Last StarFighter" ;-)


Shalom Aleikhem,
StarFighter

Posted by StarFighter | Permanent link | File under: stockfighter