Everything there is to know about Python Flask

As I was researching whether or not I wanted to go forward designing a new website using Python and Flask, I managed to find detailed information, libraries and tutorials on virtually anything you’d want to do. I figured I’d post it here for personal reference and also in-case anyone else happens to bump into it.

Using the data below you can fairly quickly build a full featured website with a database, google/facebook authentication, payment processing, and even internationalization using the very simple Python Flask framework. It can then be hosted in minutes on a very simple/cheap python hosting site like Python Anywhere. (I am not affiliated with them)

Flask is an un-opinionated micro-framework for build web apis, web services, and websites. For my use case, I found the combination of an unobtrusive framework, simple templating via built-in Jinja2 and having access to Python’s massive library, an absolute boon to development time. Lastly, if you’re new to Python, I highly recommend the free (make sure you select Community) version of PyCharm.

Flask Library Conversation
https://www.reddit.com/r/flask/

JSON/FILE UPLOAD
https://gist.github.com/yoavram/4351498
http://www.patricksoftwareblog.com/receiving-files-with-a-flask-rest-api/
https://stackoverflow.com/questions/36441398/flask-read-uploaded-json-file
https://www.techcoil.com/blog/serve-static-files-python-3-flask/
https://www.techcoil.com/blog/configuring-nginx-to-serve-files-for-a-static-website/

STORING SQL DATA
http://exploreflask.com/en/latest/storing.html
http://flask.pocoo.org/docs/1.0/tutorial/database/
https://blog.pythonanywhere.com/121/

OAUTH
https://pythonspot.com/login-to-flask-app-with-google/
https://pythonhosted.org/Flask-OAuth/
https://github.com/mitsuhiko/flask-oauth/blob/master/example/facebook.py

TOKEN AUTH
https://realpython.com/token-based-authentication-with-flask/

SSL
https://blog.miguelgrinberg.com/post/running-your-flask-application-over-https
http://flask.pocoo.org/snippets/111/
https://techtutorialsx.com/2016/12/27/9345/

PAYMENT PROCESSING
https://stripe.com/docs/checkout/flask
https://jeffknupp.com/blog/2014/01/18/python-and-flask-are-ridiculously-powerful/
https://pythonprogramming.net/paypal-flask-tutorial/
https://developers.braintreepayments.com/

SOCIAL SHARING
https://pythonhosted.org/Flask-Social/

CHARTING
https://realpython.com/web-development-with-flask-fetching-data-with-requests/
https://www.blog.pythonlibrary.org/2015/04/16/using-pygal-graphs-in-flask/

ERROR HANDLING
http://flask.pocoo.org/docs/0.12/patterns/apierrors/

INTERNATIONALIZATION
https://damyanon.net/post/flask-series-internationalization/

TEMPLATE
https://github.com/sloria/cookiecutter-flask
https://github.com/berlotto/flask-app-template
https://github.com/realpython/flask-boilerplate

SUPER TUTORIAL
http://flask.pocoo.org/docs/1.0/quickstart/
http://flask.pocoo.org/docs/1.0/patterns/packages/
( below is the best tutorial out there, start at part 1 if you’re brand-new)
https://blog.miguelgrinberg.com/post/the-flask-mega-tutorial-part-ii-templates
https://buildasaasappwithflask.com/ (NOT FREE)

Misbehaving collision and also ECS

Ah collision you naughty monkey.

On the bright side, Visual Studio 2017 performance monitoring tools are pretty cool – when they work. In this case, the tools highlighted the exact offending line of code where  the CPU is spending a comically high 50% of its time! For once, it’s not premature optimization but I was expecting this. When I hack together game engines in intense one day sessions, I tend to take brute force approaches to every solution in an effort to get something working on screen. Function that’s O(N^4) loping through every object in the game? Whatever, slap a ‘TODO’ on it and make it someone else’s problem. In this case it’s O(N^2) but what’s a power of two between friends. I first came upon the issue when I was running a test level and found an ‘AddRandomEntities()’ command in the console window mapped to F1. Curious, I kept hitting F1 until my game slowed to a craw. I looked at the dat and saw that a mere 600 collidable objects had brought the engine down.  That may seem like a lot but add a bit of bullet hell and monsters on top of more unique objects and that number comes down real quickly.

 

“A poorly coded collision function” – circa 2017

 

Fortunately, this is an easy fix in a 2D game. Subdivision of the world via a quad tree or similar structure. Really, even subdividing the screen into quadrants alone would quadruple performance. For an in-depth tutorial on build quadtrees (along with nice explanation graphics) try this. I know, I should have written something up about it here but apparently my old post was taken down from Stack Overflow for some odd reason.
Anyway, without that offending and naive collision function, the engine can render a few hundred thousand objects at 120 fps.

But while we’re talking…

Let’s talk Entity Component Systems, my new one true love. Sorry inheritance, you had your chance!  Prior to implementing one in my engine, I was hitting bottlenecks when it came to tracking, management and manipulation of traditional ‘heavy objects’. The added overhead of pulling out properties, constantly switching contexts as you attempt to navigate tree after tree of a given object was killing performance and tended to lend itself to bugs.

 

Entity Component System
“A typical ECS” – Gamasutra

 

The switch to ECS, in particular the System part came with a fairly substantial performance leap in addition to large improvements in code flow. Now instead of object hierarchies, each system is managing it’s own list of data only objects objects. Having each system iterate its list in sequence means that our current list of data being operated on is likely always hot in cache.

 

 

Each system can then operate on hundreds of thousands of data containers without a care as to who they belong to.

“They’re Everywhere!” – somedude, a prototype, 2017

For my engine, I wanted to build an ECS as opposed to an EC. There is something that is inherently elegant to me about the separation between Entity (an id), Component (data) and System (logic).
One of the tricky parts of ECS is handling cases where Systems need to operate across component types and also component lookups (which require a cast). For my approach I introduced two optimizations. First, I introduced Nodes, an idea I stole from an implementation I saw around the web. Nodes help to bridge that tiny gap between component and ‘system who needs lots of different data’. A node can hold components but also importantly, holds data which is related across all components held within. For instance here’s a simple Collision Node:
public class CollisionNode : INode
{
        public int Id { get; set; }
        public int CollidedWith { get; set; }
        public bool Checked { get; set; }
        public bool HadCollision => CollisionData.HasCollision;
        public PositionComponent Position { get; set; }
        public CollidableComponent CollisionData { get; set; }
}

The Node allows us to generate metadata about the joined objects in a lightweight way. In this way we avoid excessive lookups and lots of meta collections. We can iterate this one list and have all the info we need to take action.

To get around the casting in general that comes with ECS I opted for an enum based component type property. This isn’t the best solution, I can’t even call it an OK solution but I have yet to run into a significant issue with it. Why cast and check to ensure your casts succeeded when you can bake in the type info and grab entities matching the enumeration type. It works for me, for now.
My ECS implementation is far from the best but I’ve found that it is relatively easy to add new features and have them “just work”. After years of building inheritance hierarchies – I finally get it. If you’re not using an EC or ECS implementation in your engine, you should probably consider it!


-Jonathan

3 Things I wish I knew about F# before I started that big project

Hey everyone, today I wanted to share some insights I’ve gained while learning F# over the past few weeks/months/years (it’s been an on-again, off-again relationship). I love F# but coming from a land of large C# projects, there are some edge cases where you may find yourself tripped up if you’re relatively new to F#.

Bitshift enumerations are not supported

Coming from C#, more than a few of my enumerations followed the pattern of:

enum myEnum = 
   a = 1 << 0,
   b = 1 << 1,
   c = 1 << 2,
   d = 1 << 3

Unfortunately, using bitshift operators within a union/enumeration declaration in F# is not supported. You can however accomplish the same thing using a manually generated bit field as shown below. Easy, but slightly less maintainable.

Type myEnum = 
   a = 0b00001
   b = 0b00010
   c = 0b00100
   d = 0b01000

Null Refs will still plague you

One of the things that quickly becomes apparent when integrating your F# app into a C# ecosystem is that while F# does have a native NULL type, it will still crash spectacularly when dealing with null C# types. Consider the following:

match SomeObj.Prop with  //nullref
   | condition A -> ...
   | condition B -> ... 

Solution? Wrap your questionable calls to C# objects in a ‘toOption’ call

let toOption = function
   | null -> None
   | object -> Some

then you can use it like so..

let myVal = toOption Someobj.Prop
match myVal with
    | Some -> ...
    | None -> ...

Presto! No more null-ref worries in your elegant F# code. Also, you avoid the need for constant “if x <> null then …”.

Serialization…

One of the great things about F# is the ease of record creation. Dreams of serializing these records and sending them across the wire to your web apis can quickly be shattered when you notice all your JSON objects serialized with @suffixes. What gives? F# record members are treated like C# Fields. Their underlying fieldname is serialized by most default .NET serializers resulting in something like the following..

type myRecord {
   Name : string;
   Age:  int;
   Exp:  int;
}

being serialized as:

{ "Name@":"Gandolf", "Age@":"225", "Exp@":"15" }

Fortunately,there’s JSON.NET to the rescue! No, you won’t have to re-write your serialization logic or anything. Simply add the following attributes to your type and you’re done! The rest will be handled auto-magically.

[<CLIMutable>]
[<JsonObject(MemberSerialization=MemberSerialization.OptOut)>]

That’s all for now folks but stop by in the future as I slowly begin to wake from my stupor and flesh out this blog/site.

Cheers

So you’re an IT guy who wants to be a developer eh?

A few days ago I was searching for an initial topic to blog about, when an old friend asked me for some pointers on moving from IT to a developer role. I figured this was as good a starting point as any!

First let’s start with the secret the recruiters don’t want you to know – you don’t need to settle for an entry level position nor significant pay cut! You’ve been in IT for a while right? 2yrs? 5yrs? 10yrs? more? I made the move back in 2001 with barely 1 year of ad-hoc on-the-job development experience and went on to become a Lead Developer within 2 years! I can’t guaranteed that everyone will find it this easy, but if you’re passionate and willing to follow a few easy steps, I think you can do it!

So what are these steps? There’s three actually. I call them Immersion, Practice, and Refinement. I’ve used these three steps to do everything from Painting to Music Making to Rock Climbing. I’d say they’re more like “stages” than steps, and you’ll find yourself iterating through them repeatedly as you master a given role or skill.

Immersion

It all begins with immersion. If you’re looking to change roles, you need to immerse yourself in that new role in order to get a guaranteed drip feed of critical industry lingo and terminology. Now bear with me, you’re probably thinking “gee that’s obvious”. This is all part of avoiding the “car crash interview” and it’s part of surveying the lands and getting an idea of what are the unknown-unknowns. The great part is, you are probably already performing this step without even thinking about it. If so, congrats! You’re already 1 for 3! The key to immersion is to become a part of the environments which developers do. Hang out on industry blogs, find a local meetup or free conference, lurk in a development Slack channel, hangout on the Stack Exchange Software Engineering Channel, or even scan through Reddit’s many developer forums. You want to become familiar and used to the common terminology you’re going to initially encounter. Learning the lingo, even at a surface level, helps to improve your confidence and prime your mind for better understanding when you are ready to engage those topics fully in the future. If I were to say “what sort of interfaces and design patterns might you choose to construct a library book management app”, you don’t want to be looking around like a kid lost in the woods. After immersing yourself it’ll be impossible to not have at least heard these terms several times in passing. You might even know that ‘interfaces are how we define contracts in software’ and ‘design patterns are re-usable software designs for common problems’. You might not be able to answer questions for a “Lead programmer” role, but you’ll know enough to say* “My experience with design patterns is limited but I’d….”*. The key is to get a better understanding of what you do and do not know. Once this becomes clear, you can approach both learning and interviews in a cool an honest way.

Practice

Next comes practice. This stage is simple but critical. It’s time to build an app! You’ll want to set yourself up with a fresh GitHub account and create a free, personal code repository.  Think of a task that you perform on your computer, or a common automation task that you perform by other means and let that be the starting point for your initial project. My first serious app was a network scanner which used multi-threading to reach out, ping, and record the responses on a given class C network. Simple. If that sounds difficult, don’t worry,  in a few short weeks of practice and immersion, you’ll be able to write that same app — even if it isn’t perhaps the cleanest and most object oriented code ever.

Having fresh code in your GitHub accomplishes several things. First, it proves you’re serious about wanting to be a developer. Second, it shows that you have at least a basic understanding of the most popular code repository on the planet. Third, it will be your own personal motivation. Ask any coder and they will tell you that starting a new project can be tricky but once it gets going, you’ll end up coding into the wee hours of the morning as you think of new features and ways to optimize, refactor and improve your code. Once that happens (if it hasn’t already), you will have ignited an insatiable desire to develop ever more performant and elegant code — the learning will come as a byproduct.

Refinement

The last stage is what I like to call refinement and I’ll tell you right now, it goes on indefinitely. This is the constant process by which you’ll take the things you’ve learned while immersing yourself and practicing, and dig deeper. “This SOLID acronym seems to be mentioned a lot, what’s that”? These are the questions you’ll start digging into. Now it’s time to open a book or browser, and learn more about concepts that you may only know by name. If a concept still seems ‘clear as mud’ after reading about it, lay it down, and loop back around to another concept. It takes years to gain full and deep master of all the myriad concepts associated with software development. That being said, within a few weeks of immersion, refinement and practice, you’ll have the skills to talk smartly and demonstrate your knowledge of common software engineering concepts!

Good luck!

Jonathan