Einstein wasn’t wrong when he said the only thing that got in the way of his learning was his education. Learning how to think — creatively to solve problems, to find dots and connect them, are skills that are hard to find, hard to develop, especially when we can get into a trance with technology.

I think we’re most alive when we create.

So that’s all I’d do. Create, create, create. Solutions for lots of problems, in lots of ways, to realize that everyone just wants to answer one question with their solutions nearly every time. “Where is everything at?”

Learning how, and why to think from multiple perspectives when looking at solving a problem is infinitely more important than the tool(s) you pick to use them.

The tools you pick will be better based on the clarity and your ability to boil things down to their essence. Tools that increase clarity and decrease confusion are good. Tools that let you (and users) get more done with less effort, are good.

Helping people make the world a better place by letting the system manage the details, and letting people manage the system (and relationships with others) is what can empower software to change the world for the better, instead of dealing with the 70% of failing projects out there.If

February was a bit of a busy month for me so I wrote some content in advance to publish for sure, and to add more when I had the chance..

Evidently, PHP and wordpress do not work out of the box to automaticaly publish scheduled content.

Maybe I should go to a ColdFusion blog.

In any event, my apologies, I’ll post the rest and it should show up on the RSS feeds.

In building secure applications that let me sleep at night, I have learnt a few things:

1. Security is relative. You must keep up, and ahead of the curve.  Your thinking and design must be steps ahead of what’s happening and what may be possible in the near future.  In the end, if someone wants to, given enough of their time, resources and money (more than what you have spent) it, still, conceivably, is possible that security may be compromised.
2. Security vs. Convenience. Unless you are a expert of making complex security disappear “under the covers”, there is still a fundamental  acknowledgment that has to be made.  The more security you implement in an application, the performance, and usability of that application becomes less than simple.
3. The path of least resistance. Instead of building a big, thick wall of security, consider building many, thinner walls. Just like someone who asks you a question instead of googling it, someone wanting access to a system will want to do so the quickest way possible.  If they are out to find the next outpost, they will move on if there is layer after layer of security, like an onion skin.

   Why?  Most unscrupulous attempts to access a system aren’t really about you.  Most valuable is the most generic.  Your server resources to be illegitimately used to send out spam, etc, and not necessarily for what your application itself does.  The reality is most attempts to break into your site will be automated scripts/botnets that sweep the entire internet.  While you can build mega security features, a lot can be said by good design and putting up multiple “walls” instead of one big “wall”.  Design your application to inherently check and enforce security as much as it can internally.  Package it with smart public facing interfaces, properly hardened and tested with client and server side scrubbing and validation, and we have a start.

4. Passwords be safe. Do your best, reasonably to secure the single, most vulnerable part of your system.  Relative to the sensitivity of what is behind the password, implement password security measures.  Logarithmic delays between failed attempts.  Log and report invalid attempts. Encrypt your passwords.

   One of the biggest things you can do if/where necessary is to store your passwords securely.  This is a great article on How to safely store a password.

If you have any ideas to add to this list, please leave them in the comments.. I’ll update the list!

The past few weeks I’ve been thinking about how I’d like to continue to build in 2010.

Whether you are on the business side of technology, or the implementation side, there are some common elements to all projects no matter the viewpoint.

Great software systems:

• Make users great. Make your users awesome at what they do when they use your software.  If you don’t, you have a dud.
• Are mature. People manage the system, the system manages the details.    Immature: People make their own systems, manage their own details, nothing is connected, or consistent.
• Understand the data is the system: The data is the system to the end-user.  Not your software.  The software is merely packaging to the information they need.
• Understand the integrity of the data: What is the data. What does it mean.  What states does it exist in.  How does it interact with other data.  Why is this important?
• Keep the edge: Software is built around the competitive advantage of magnifying and fueling the existing best practices of “This is how we do it here”.  Don’t lose it by doing it your way.
• Are invisible: “Don’t make me think” when I use it.
• Self-Monitor: Bring things to my attention
• Master the Complex: It ’s easy to make things complex.  It is hard to make complexity into something simple.  Understand that complexity is not the issue most of the time.  It’s confusion.

Maybe this will become a growing list!  Add your own below!

I build a Web application when I don’t want to:

- be supporting a thousand environments each with their own quirks. Specifically, viruses, trojans, software interfering, and making it work the same everywhere.

- worry about applying upgrades and taking lots of calls

- dealing with clients who lose their data

I pick a thick client when the computation is intensive per transaction, that would be simultaneous, or there is significant data transfer per transaction.

I like fixing everyone’s problems with one upgrade. It might not be for everyone, but it’s where the quality of my life and those I work for goes up. Making a web app work on a few different browsers may be easier than a ton of different operating systems in different conditions.

With the advent of Flex/Air you can provide an entire application experience in the browser. The browser is becoming the universal interface, whether it’s installed locally or in the cloud.

Web apps have their cons too. I just am more motivated to build web apps as the pro’s seem to outweigh the cons for the projects I am choosing.

One thing I’ve found myself doing more and more in ColdFusion is using it for it’s integration powers to parse data, or do batch performing tasks.  With it’s Java underbelly, ColdFusion does most things great, and if you want even more performance, tie into the Java layer!

I come from the group of people who pursue the dream that computers and technology should make people’s lives easier.  That’s why ruthless automation is at the center of most startups, or businesses that are growing.  Keep people dealing with people — what they do best, and let your system handle the details for you!

1) Maintain a file upload table (Parent table). For every file you upload you should be able to keep a list of each file and what status it is in (uploaded, processed, unprocessed)

2) Temp table to store all the rows of the data file. (child table) Import the entire data file into a temporary table. Attempting to do it all in memory will inevitably lead to some errors. Each row in this table will link to a file upload table entry above.

3) Maintain a processing status – For each row of the datafile you bring in, set a “process/unprocessed” tag. This way if it breaks, you can start from where you left off. As you run through each line, set it to be “processed”.

4) Transaction – use cftransaction if possible to commit all of it at once, or at least one line at a time (with your 5 queries). That way if something goes boom, you don’t have one row of data that is half computed/processed/updated/tested.

5) Once you’re done processing, set the file name entry in the table in step 1 to be “processed”

By using the approach above, if something fails, you can set it to start where it left off, or at least have a clearer path of where to start investigating, or worst case clean up in your data. You will have a clear way of displaying to the user the status of the current upload processing, where it’s at, and where it left off if there was an error.

If you have any questions, let me know.

Other thoughts:

1. You can increase timeouts, give the VM more memory, put it in 64 bit but all of those will only increase the capacity of your system so much. It’s a good idea to do these per call and do it in conjunction with the above.
2. Java has some neat file processing libraries that are available as CFCS. if you run into a lot of issues with speed, you can use one of those to read it into a variable and then into the database
3. If you are playing with XML, do not use coldfusion’s xml parsing. It works well for smaller files and has fits when things get bigger. There are several cfc’s written out there (check riaforge, etc) that wrap some excellent java libraries for parsing xml data. You can then create a cfquery manually if need be with this data.