npm bash-conf, for sharing configs between bash and node

API keys, DB authentication, passwords, system paths and other things of that nature dont belong in a Git repo.

They belong in a lovely config file created by an install script run after cloning a repo. That’s how I try and do things these days anyway.

I use a lot of bash scripting for big queries and file manipulation, often creating strings and files in Coldfusion and saving them to Amazon S3 or the HDD for a bash script to sort out later.

So these install scripts ask for db credentials, API keys etc and you type your responses in to match the environment you’re on (production, staging, dev) and the bash scripts work away after importing the data.

I’ve started using node js for these back end processes more and more frequently now, allowing me to develop more complex scripts that are easier to debug, easier to maintain and most of all means my team mates can work on them as well!

In order to avoid having to have another JS friendly config file for my node scripts to read and also to avoid passing data in when the scripts are called I’ve written an npm module that can read the same config file bash uses.

https://www.npmjs.com/package/bash-conf

To quickly summarise how it works, I’ll just dump the usage snippet from the README. You just have to give it the path to a config file which is just a list of simple FOO=BAR style declarations

var path = process.argv[ 2 ],
BashConf = require('bash-conf'),
bashConf = new BashConf();

bashConf
.read( path )
.then(function( data ) {
console.log( 'what is foo', data.FOO );
console.log( 'this should be empty -> [', data.EMPTY_VAR, ']' );
})
.catch( function( err ) {
console.log( err );
});

If you want more info, get in touch, use Github issues or RTFM!
Also get in touch if the manual needs updating…

ConsoleFancy – my first npm module

First off, I’m not really sure if it’s a module, or a package, or a repo or what. Maybe it’s all three. I’ll find out one day.

More importantly I’ve added to npm! You can install a package that I’ve written right now – it was ridiculously easy to do.

vi test.js

// create the following in test.js
// var ConsoleFancy = require('console-fancy'),
// consoleFancy = new ConsoleFancy();
// consoleFancy.header('hello');

npm install console-fancy
node test.js

After all that hard work you should see hello wrapped in the default top level borders – currently a lot of # symbols. You can switch it to consoleFancy.header('hello',2); for the second level header or when you init the object you can pass it more levels and specify the border, padding and edges. I’ll expand the docs to show how to do that when I get some more time.

It all came about as I was writing a Node.js app to download and process Apache log files from my Load Balancer, when I got frustrated at the amount of times I’d written console.log(). It was all blending into one output and was hard to read what was going on. I wanted to break the output up so I could clearly see major points in my output.

So I wrote a function that I could pass my log message to, as well as a level heading, like an <h1> or <h2>. This worked fine but I wanted to be able to re-use it and share it with my colleagues.

I created a new script and included that with require('./ConsoleFancy.js') and then thought “How do I get this into npm? Can I add this, and just install it like all the other packages?”

Well the answer is yes. It’s incredibly easy. It’s all explained here:
https://docs.npmjs.com/getting-started/publishing-npm-packages

In short: Once I’d signed up at npmjs.com, as I’d already created my node module/package thing, I just ran the following to get it published:

// this created the package.json for me - it asks you questions, you answer them
npm init

// log into the npm site, so it publishes to your account
npm login

// then you publish it... if there's any problems it errors and you can try again
npm publish

The process gets most of the information it needs from the package.json. When I ran the init command it asked me what the Github repo was, so I actually cancelled the process, created a repo, pushed it all live and then started again.

Adding the Git repo means it’s linked from the npmjs site, so other people can contribute, submit pull requests, issues etc.

When you update the module, once you’ve made you changes, commited your code, you run the following command:

// patch can be replaced with minor or major
// https://docs.npmjs.com/getting-started/semantic-versioning
npm version patch

This command is brilliant, it does 2 things:

  1. Updates the package.json file, incrementing the version number based on your patch/minor/major keyword
  2. Creates a Git tag for your repo so you can get back to this release commit – the tag matches the version number

Once you’ve created your new version you just run npm publish again and it not only pushes it npmjs.com, but also pushes the new commit and tag to Github!

It’s also worth making a README.md file file (or whatever format you like for your README) as this gets pulled into your npmjs page – without it you just get a blank page with the project name on it.

So if you find yourself writing utilities and tools for Node.js, and you’re considering sharing them then go ahead and do it. It’s so simple, it’ll make your life easier in your next project, you’ll be helping others and if it picks up you can get some help to improve it.

Just don’t forget to check if someone else has already written it… I did look, but I bet someone’s already written a ConsoleFancier.

Project links:

Professional errors

I am a professional web developer. I get paid to make websites and this has been the case for the last 5 years.

As each year goes by I get better and better at my job, my skill set expands, I get better at debugging and fixing broken code, I produce code that needs less debugging and less fixing, code takes less time to write and it can do a whole lot more than the older stuff.

But I do of course still make some mistakes. Take these 2 little gems from today.

I’d written html like this:

<div class="#idName">bla bla</div>

Then I couldn’t work out why this rule wasn’t being applied:

#idName {
color: red;
}

For my next mistake, I did something like this:

<div class="ting fade tingBigger">bla bla</div>

And again failed to understand the problems with this CSS:

#tingBigger {
width: 100%
}

Luckily, I’d also done lots of awesome excellent CFScript and Javascript to make up for it throughout the day.

CSS after/content

I learnt some mega useful CSS today, which I feel like I should have known already.

I found this out from a Treehouse video, talking about print styles sheets. Treehouse is an excellent website full of useful and well produced video tutorials covering a wide range of development, design and computing topics, including Graphic Design, HTML, iOS Coding, Ruby and lots, lots more.

The particular video I was watching was about best practices in print style sheets for websites. Using the combination of the :after psuedo-element and content property, you can get your stylesheet to show the full URL of a link when printing out a document.

So taking the following link, which will just appear like this: Best site ever


Best site ever

… you can then apply CSS like the following to produce this:


a:after {
content: "
" attr(href);
}

Best site ever

http://skeater.co.uk

There’s also the “before” psuedo-element, which obviously does similar things to after.

Merge sort in Coldfusion

For far too long I’ve been using Bubble sorts, they’re very basic and very slow but mainly I’ve been sorting tiny little arrays or lists and the server just don’t care about it.

But that’s not the proper way of doing things. Now I don’t really understand sorting algorithms all that much so I use this site to work out which one I want to use, plus a little bit of wiki:

Sorting-algorithms.com

I’ve decided to use merge sort from now on mainly because it’s quite fast and although it generally seems to be a tiny bit slower than a shell or heap sort, it is stable which shell and heap are not. I don’t really know what that means but it sounds safe! Also it works much better on large sets than any of the simple algorithms.

The algorithm is more complex than a Bubble sort and although I understand the gist of it I decided to find someone elses code and use that. So I did.

Simon Horwith’s blog entry on search algorithms

This guy has made a fantastic little demo app, very comprehensive, with 6 different sortin algorithms in it. You can download the zip and drop it into a server and test the speeds of each algorithm. Then the code is there for you to play with, or as Simon put it: I look forward to hearing how people put them to use as well as any other findings with them.

I’ve tweaked his code a bit, so it will take an array of structures/objects, and sort by any publically accessible field of the that structure. Here’s how I’ve used it in the test page:

sort = new mergesort();

a = {};
a.name = 'Pete';
a.age = 27;

b = {};
b.name = 'Holly';
b.age = 22;

c = {};
c.name = 'Jesus';
c.age = 2012;

peeps = [a,b,c];

writeoutput('

Original array

');
writedump(peeps);

// sort peeps array, by oject.name into Descending (default) order
peeps = sort.sortArrayByFieldAscending(peeps,'name');

writeoutput('

Sorted by name in descending (default) order

');
writedump(peeps);

// sort peeps array, by oject.age into Ascencing order
peeps = sort.sortArrayByFieldAscending(peeps,'age',true);

writeoutput('

Sorted by age in ascending order

');
writedump(peeps);

You can view the live demo of it working here http://skeater.co.uk/samples/mergesort.cfm and I’ve also attached a .zip of the .cfm and .cfc.

The ZIP link above doesn’t work at the moment, I’ll have it fixed shortly!

Thanks to Simon Horwith, and hopefully someone else will find this useful!

Precisely the same Railo mistake as last time

In the past I’ve had problems with my blog working on mobiles. Something to do with the way it’s set up for Coldfusion but I’m running Railo.

I blogged about it here: My very few BlogCFC woes

So when I switched across to my new server recently, I forgot to make the changes to get it running on my new server. I have in fact done something slightly differently this time though, I’ve changed the /railo/tomcat/conf/web.xml file instead of the one in the instance of Skeater.co.uk, so the changes are global.


GlobalCFMLServlet
*.cfm
*.cfml
*.cfc

/index.cfm/ *
/default.cfm/ *
/post.cfm/ *
/archive.cfm/ *
/blog.cfm/ *
/page.cfm/ *

/mobile/index.cfm/ *
/rewrite.cfm/ *

Note: once again, the white space between the slash, /, and the star, *, is just
to stop the blog trying to use it as a comment.

So thanks Sparky for pointing out the error, and no thanks to you Pete, for being an idiot and not correcting the problem that you’ve had before so you should have known! Dumb ass.

Editing all the links in a page

I’m not sure if this is a duplicate post of not… I couldn’t find the original one, so here goes anyway.

We had the situation at work where we wanted to find all the links in a news article, in the back end editing system, and send them all to something like bit.ly to shorten them, so make it less likely that the article hit the maximum word count.

This might have been a forum post actually, anyway, that’s besides the point.

So I came up with a quick example to work out how to do it. Instead of using a URL shortening API for the demo I’ve just reversed the hrefs. But it would be easy to swap them that code out for bit.ly or something similar.

Edit HREF's demo

This is totally copied from Ben Nadel's blog, an idea which he got in turn from A Book Apart: Mobile First, but I thought it was worth mentioning again.


#longText#

// find the a tags
arrHref = reMatchNoCase(']*>',longText);

// get the array length and loop over
cntHref = arrayLen(arrHref);
for(h = 1; h LTE cntHref; h++) {

// get the individual href,
thisHref = arrHref[h];
thisHref = reReplaceNoCase(thisHref,'
','1');

// reverse it, bit.ly it, whatever takes you're fancy
reverseHref = reverse(thisHref);

// replace the original href with the new one
longText = replaceNoCase(longText,thisHref,reverseHref);
}



#longText#

You can see the live sample here: http://skeater.co.uk/samples/hrefEdit.cfm

It might not be completely robust, it might need updating for production use but the concept is there and it works.

MySQL update/insert/delete rows affected in CFScript

There’s a way of getting the rows affected by an update, insert or delete MySQL query in CFScript.

I didn’t realise this until yesterday, and I’ve been doing this for a while now. I really should read more release notes.

I’ll update my http://skeater.co.uk/page.cfm/CFScript-Snippets page to show this, but I thought it was worth posting about anyway

// create query string
strSelectRows = 'DELETE FROM table WHERE col = "potato"';

// create query, assign datasource, assign query string, execute and capture returned result
qrySelectRows= new Query();
qrySelectRows.setDataSource(APPLICATION.dsn);
qrySelectRows.setSQL(strSelectRows);
qrySelectRows= qrySelectRows.Execute().getResult();

// dump results out
writeDump(qrySelectRows.getPrefix().recordcount);
writeDump(qrySelectRows.getPrefix());

Check the demo here to see the tasty results. I’ve also shown the whole prefix component in the demo.

Automatically bit.ly-ing URLS in a text block

Gosh, what a long title. I don’t like writing blog post titles, but that one seems to make sense.

I wrote this code the other day as a kind of demo for an idea at work.

We allow some HTML in forum posts at work, but if a user puts in loads of long URLs then they hit the post limit quickly. So we wanted to write some code that would scan through a post, find the URLs, shorten them, and then cap the post to it’s limit.

Instead of using a bit.ly API here (other URL shortening services are available) I’ve just reversed the href, so http://site.com becomes moc.etis.//:ptth… which was quite a lot of fun in itself.

Edit HREF's demo

This is totally copied from Ben Nadel's blog, an idea which he got in turn from A Book Apart: Mobile First, but I thought it was worth mentioning again.

arrHref = reMatchNoCase(']*>',longText);

writeDump(arrHref);

cntHref = arrayLen(arrHref);

for(h = 1; h <= cntHref; h++) {
thisHref = arrHref[h];
thisHref = reReplaceNoCase(thisHref,'
','1');
reverseHref = reverse(thisHref);

longText = replaceNoCase(longText,thisHref,reverseHref);
}

#longText#

Simple passwordless SSH tutorial

It took me many attempts to work out password-less ssh, mainly because a lot of internet guides are not written that well and following them step by step didn’t work.

Then I found one that did work! Woohoo!

So I’ve copied and pasted it here… and regrettably forgotten the link I found it from. If anyone recognizes this then I’ll gladly reference it to the correct source.

The guide makes use of local$ and remote$, this is to indicate if you are typing on your local or remote server.

Create the private key


local$ ssh-keygen -t rsa (accept default file locations and create a password for your private key)
local$ scp ~/.ssh/id_rsa.pub user@remote:/home/user/
local$ ssh user@remote (enter normal root password)
remote$ cat ~/id_rsa.pub >> ~/.ssh/authorized_keys
remote$ chmod 600 ~/.ssh/authorized_keys
remote$ exit
local$ ssh user@remote (at this point you shold be asked to enter the password for your private key)

Add private key password

The password for the private key needs storing in your machine. This is encrypted.

local$ ssh-agent bash
local$ ssh-add ~/.ssh/id_rsa
local$ ssh user@remote (you should no longer be asked for a password)

Note: in the code examples above, don’t type the stuff in brackets

Note 2: I think this was the blog I got the entry from, but it doesn’t work any more so I can’t be sure… I definitely looked there before I found it anyway: http://blogs.translucentcode.org/mick/archives/000230.html