Lynch Consulting Blog - Open Source

ServerStats mentioned on CFHour

Tue, 29 Jun 2010 13:34:00 --1000

Just came across a mention of a little project that I did back in 2006 in the most recent CFHour podcast - Show #58 - Monitoring, Debugging, and Guests. (Mention is at approx 53 minutes). The ServerStats code wraps up a few undocumented methods for CF to make it easy to get information on the number of sessions active on a server as well as some basic information on memory usage. Thanks to Charlie Arehart for mentioning it - keep up the good work. I must have a look at updating it to support Railo when I get a moment. Cheers, Mark

Howto find files newer than a specific date using command line

Mon, 10 May 2010 23:19:00 --1000

While doing some server admin tasks the other day I needed to find all the files newer than a certain date. Using just the command line tools it was relatively simple but not obvious, so this is a not to self. The find utility has an option to find a file newer than another file. By creating and empty file with a specific creation date we can do the search:


touch timestamp -d 2010-01-01

To show all files newer than 2010-01-01 use:


find . -newer timestamp

Or to create a tar archive of them use xargs like so:


find . -newer timestamp | xargs tar -rf /root/filesnewerthan-2010-01-01.tar

Easy. Mark

Flex 3 to Flex 4 Migration Howto

Mon, 22 Mar 2010 09:50:00 --1000

I recently completed migrating an application from Flex3 to Flex4, and for my own reference, here's how I did it. Note - this was done with flexsdk-4.0.0.13875, which was the most recent stable build at the time.

Phase 1: Get Flex3 code compiling with Flex4

Fix up stylesheets:

If you have a current Flex3 project and you are migrating to Flex4 you need to add the following lines at the top of your stylesheet.


@namespace "library://ns.adobe.com/flex/mx";
@namespace s "library://ns.adobe.com/flex/spark";

This makes the default non-namespaced items in your stylesheet refer to the MX components. eg:


Button{fontSize:18;}

And to style up spark buttons you simply use:


s|Button{fontSize:18;}

To statically link or not?

Flex 4 defaults to dynamically linking runtime shared libraries. This produces smaller files, but if you are making desktop type applications or developing code while not connected to the internet you'll probably want to statically link the files. Add this to your compiler option:


-static-link-runtime-shared-libraries=true

You now should have a happily compiling app, however, to use all the new goodness of flex 4 there are still some more steps to be done.

Phase 2: Using all of flex 4

Update namespace to 2009

To update the namespace references for flex 4 I did the following search and replace:


Replace:
xmlns:mx="http://www.adobe.com/2006/mxml"
With:
xmlns:fx="http://ns.adobe.com/mxml/2009" xmlns:mx="library://ns.adobe.com/flex/mx" xmlns:s="library://ns.adobe.com/flex/spark"

Replace mx with fx namespace


  and 
becomes:
  and

Similarly for other tags like this:


  to 
  to

Add Declarations for non visual components

Wrap tag around non visual compoents. The compiler will complain about all these errors so just follow through and wrap them in

tags until all the errors go away.

Migrate States

Finally, at least for the project I migrated, I needed to convert all the state tags to the new format of inline attributes. This shows up as the following error:


Error: State overrides may no longer be explicitly declared. The legacy states syntax has been deprecated.

So if you had something like this before:


 
  
   
            
... snip ...

It would now look something like this:


 
... snip ...

Migrate StyleManager References

Stylemanager is not called as a global class any more and you need to use fully qualified references so:


thisImage.source = StyleManager.getStyleDeclaration("Image").getStyle("brokenImageSkin");

becomes:


thisImage.source = styleManager.getStyleDeclaration("mx.controls.Image").getStyle("brokenImageSkin");

When these have all been converted, your application should compile. Then you can start leveraging the new functionality of flex4 and begin migrating your components to spark if necessary.

MySQL 5.1 logging changes - Log to DB and runtime config

Fri, 29 Jan 2010 08:15:00 --1000

While browsing around the MySQL site last night I discovered a number of nice new features of mysql 5.1 that relate to logging. These are:

Logging to DB instead of log files
Runtime configuration of logging.

Logging to DB instead of log files

Coming from a web development background rather than a sysadmin background I'm far more comfortable manipulating and analysing data using SQL. So to be able to log all the queries or just the slow queries for an application to the db during application development or load testing is a huge benefit. To enable logging to DB you can add the following to your my.cnf


log_output = TABLE

The logs will be written to the 'slow_log' and 'general_log' tables in the mysql database. Note - logging to tables has more overhead than logging to file, so would suggest using it primarily for development purposes. Full details of the options are on the mysql manual on log tables

Runtime configuration of logging.

This allows you to turn on and off logging without restarting MySQL - which just saves a little bit of time and makes it much nicer for debugging problems. To turn on the logging of all queries run:


SET GLOBAL general_log = 'ON';

And for just the slow query log:


SET GLOBAL slow_query_log = 'ON';

And to turn them both off use:


SET GLOBAL general_log = 'OFF';
SET GLOBAL slow_query_log = 'OFF';

If you also want to see queries not using indexes in the slow query log you can set the following variable:


SET GLOBAL log_queries_not_using_indexes = 'ON';

Hope it helps, Mark

Jmeter over SSH Socks proxy

Thu, 07 Jan 2010 10:25:00 --1000

I've been doing some testing recently where I need to connect via SSH server to a remote network to run some load testing. To do this I used a SSH sock proxy like I have previously blogged about. So I fired this up so that I could review the site I wanted to look at. It worked a charm through firefox but there is no where to set up the proxy in jmeter. To make it work you need to let the JVM know what proxy to use like so:


java -DsocksProxyHost=localhost -DsocksProxyPort=8080 -jar ApacheJMeter.jar

No jmeter will use the socks proxy on port 8080 on my local machine. Nice.

Upcoming book review - Tomcat 6 Developer's Guide

Mon, 21 Dec 2009 21:12:00 --1000

I've just received a copy of the Tomcat 6 Developer's Guide from packt publishing to review. It's nice timing as I've been working with Tomcat 6 a bit lately and in the new year plan to move some of our production systems over to running Railo on top of Tomcat. After the extremely busy year Learnosity has had I'm looking forward to reading a few books over the break and coming back in the New Year with lots more ideas and technology to implement.

Learnosity are looking for a Web Ninja at a Mid to Senior level

Fri, 04 Dec 2009 16:03:00 --1000

4 December 2009, Learnosity are looking for a Web Ninja at a Mid to Senior level.

About Learnosity

Learnosity develop cutting edge tools for teachers and educators. Our flagship product Learnosity Voice uses the telephone to enable language students and teachers to interact on a one to one level. Our service:

Makes it practical for students to practice Oral and Aural skills
Is efficient and effective for teachers, as they can listen to each student individually at a time to suit them
Can be used for homework assignments or “High Stakes Assessments”

We are continuing to grow our core development team and we need another great developer to help us keep up with demand.

We need someone who can:

Use Javascript or Actionscript to create great user interfaces
Develop highly scalable web applications
Cut code with the best in the world

You will also need to be:

energetic with a butt kicking attitude
ready to create cutting edge web 2.0 apps
keen to continue learning new technologies
able to have a conversation with non technical people

You'll need:

3 or more years of programming experience
Expert in at least on Client side language (Actionscript or AJAX)
Expert in at least one Server side language (eg PHP, Java, ColdFusion, etc)
Understanding of Object Oriented design
Understanding of XHTML and CSS

It would be good if you have:

A degree in Computer Science, Engineering or similar.
been working with open source tools
been playing around with iPhone/Android applications
experience with some of Linux/VOIP/SIP/Asterisk/Jabber/XMPP

This is a full time role and you will be working in a casual workplace with flexible hours in the Sydney CBD. If this sounds like the job for you, email a covering letter explaining why you'll be great and your resume to mark@learnosity.com - no agencies please.

Normalising audio with sox

Wed, 25 Nov 2009 22:48:00 --1000

Update - as of sox 14.3.0 it is much simpler than described below - simply do: sox --norm before.wav after.wav Seem to be on a roll with Sox tonight. Also figured out how to normalise audio files nicely. The first thing to do with sox is to get it to calculate the max volume adjustment possible:


sox before.wav -n stat -v

This will return a number like: 4.234 You can then call sox again using this number:


sox -v 4.234 before.wav after.wav

And if you're feeling particularly good you can put the whole command on a single line as follows:


sox -v `sox before.wav -n stat -v 2>&1` before.wav after.wav

Easy when you know how.

Removing silence from audio using sox

Wed, 25 Nov 2009 22:11:00 --1000

I've had a look at this a few times before and each time I've gotten a little bit further and then gotten frustrated and failed. However, this time I finally managed to get past all the hurdles and jump through all the hoops to make this work. Scenario is a follows. Audio file is comes in in alaw or ulaw format and we want to trim any silence before and after it. First step is to convert it from alaw to pcm encoded wav file as SOX has issues filtering silence in alaw files


sox -t alaw in.alaw -c1 -2 -r8000 -e signed-integer temp.wav

Then we do the trimming in one fell swoop.


sox temp.wav out.wav silence 1 0.1 0.1% reverse silence 1 0.1 0.1% reverse

A huge thanks to this article for giving such a thorough overview of the how silence detection works in sox. Too easy, when you know how :-)

Chinese characters not working cfdocument for PDF exports - fixed

Mon, 26 Oct 2009 16:38:00 --1000

I've just spent quite a few hours grappling with an annoying PDF export issue where the font's were not displaying correctly. This is on a CFMX 7 (7.0.2) on Ubuntu Linux system. All the DB storage and application was using UTF8 throughout, but the PDF exports were showing nothing when they should have been showing Chinese characters. To fix this up you need to do the following:

1. Need fonts installed on system:


sudo apt-get install ttf-arphic-bsmi00lp ttf-arphic-gbsn00lp

2. Need fonts installed in CFAdmin

Go to CFAdmin and Font Management
Select directory /usr/share/fonts/truetype/arphic
Select "Add" and you should see 2 new fonts added.

3: CFdocument Hotfix for CF7.0.2

The CFDocument hotfix for 7.0.2 provides some essential fixes and makes this work - Follow the instructions to install: http://kb2.adobe.com/cps/402/kb402093.html

Configure cffont.properties file

The final step is to set the cffont.properties file so that it will check additional font sets for any characters it doesn't know about - i.e. Chinese characters. Each line in the file is like a CSS font-family rule, i.e. it will start at the left and look for the correct font to render the text. I appended the two extra font names to ensure they are checked, so the file looks like this:


defaultbasefont=
dialog=Arial, Helvetica, AR PL SungtiL GB,AR PL Mingti2L Big5
dialog.bold=Arial Bold, Helvetica-Bold, AR PL SungtiL GB,AR PL Mingti2L Big5
dialog.italic=Arial Italic, Helvetica-Oblique, AR PL SungtiL GB, AR PL Mingti2L Big5
dialog.bolditalic=Arial Bold Italic, Helvetica-BoldOblique, AR PL SungtiL GB, AR PL Mingti2L Big5
dialoginput=Courier New, Courier, AR PL SungtiL GB, AR PL Mingti2L Big5
dialoginput.bold=Courier New Bold, Courier-Bold, AR PL SungtiL GB, AR PL Mingti2L Big5
dialoginput.italic=Courier New Italic, Courier-Oblique, AR PL SungtiL GB, AR PL Mingti2L Big5
dialoginput.bolditalic=Courier New Bold Italic, Courier-BoldOblique, AR PL SungtiL GB, AR PL Mingti2L Big5
serif=Times New Roman, Times-Roman, AR PL SungtiL GB, AR PL Mingti2L Big5
serif.bold=Times New Roman Bold, Times-Bold, AR PL SungtiL GB, AR PL Mingti2L Big5
serif.italic=Times New Roman Italic, Times-Italic, AR PL SungtiL GB, AR PL Mingti2L Big5
serif.bolditalic=Times New Roman Bold Italic, Times-BoldItalic, AR PL SungtiL GB, AR PL Mingti2L Big5
sansserif=Arial, Helvetica, AR PL SungtiL GB, AR PL Mingti2L Big5
sansserif.bold=Arial Bold, Helvetica-Bold, AR PL SungtiL GB, AR PL Mingti2L Big5
sansserif.italic=Arial Italic, Helvetica-Oblique, AR PL SungtiL GB, AR PL Mingti2L Big5
sansserif.bolditalic=Arial Bold Italic, Helvetica-BoldOblique, AR PL SungtiL GB, AR PL Mingti2L Big5
monospaced=Courier New, Courier, AR PL SungtiL GB, AR PL Mingti2L Big5
monospaced.bold=Courier New Bold, Courier-Bold, AR PL SungtiL GB, AR PL Mingti2L Big5
monospaced.italic=Courier New Italic, Courier-Oblique, AR PL SungtiL GB, AR PL Mingti2L Big5
monospaced.bolditalic=Courier New Bold Italic, Courier-BoldOblique, AR PL SungtiL GB, AR PL Mingti2L Big5

The cffont.properties file is located in /opt/jrun4/servers/{instance name}/cfusion.ear/cfusion.war/WEB-INF/cfusion/lib on the jrun multiserver version.

5. Restart CF

Once you restart CF you documents should be coming out with their Chinese fonts intact. I'm sure there are additional fonts to add to get it working natively with all the other asian languages, so I'll add to this as and when I need/discover them. Please feel free to post a comment if you find any more font's that should be added to the list.

Two quick server tips

Sat, 15 Aug 2009 08:12:00 --1000

Here are two quick tips that I've recently found for server admins:

Automatically fixing file system errors
Ignoring directories from updatedb

Automatically fixing file system errors

If you have a remote server, i.e. at a data center 10km or 1000km away this should prevent some panic when rebooting the server remotely. Ext2/3 will do a filesystem check after a certain number of reboots or time. Most of the time any errors are fixed automatically, but certain errors by default require the root shell and the administrator to fix them. I've seen a few of these happen, but I have always replied yes to the prompts as I don't know enough about file systems to fix it I said no. So, to prevent the need to rush to a data center and plug in a keyboard and mouse just to press the "Y" key there is and option to automatically assume yes. On Ubuntu in the file /etc/default/rcS you need to change the following:


FSCKFIX=no
to 
FSCKFIX=yes

Ignoring directories from updatedb

If you have a backup server you may not want it to update the locate db for all your backup files, as it can take a very long time. To tell locate to ignore a directory you need to add it to the PRUNEPATHS line in the /etc/updatedb.conf file like so:


PRUNE_BIND_MOUNTS="yes"
PRUNEPATHS="/tmp /var/spool /media /srv"
PRUNEFS="NFS nfs nfs4 afs binfmt_misc proc smbfs autofs iso9660 ncpfs coda devpts ftpfs devfs mfs shfs sysfs cifs lustre_lite tmpfs usbfs udf rpc_pipefs"

In the above code I have added the /srv entry to make it ignore all my backups which are held under the /srv directory. Note: These tips were tested on Ubuntu linux, other distributions will have similar functionality but the file locations may vary. Cheers, Mark

HOWTO find files that are not world readable

Mon, 18 May 2009 21:47:00 --1000

I ran across some permissions errors on content that some users uploaded to a website. The files uploaded were not world readable so were coming up as forbidden: This simple find command shows the offending files:


/usr/bin/find . -type f ! -perm -004

A quickie but a goodie. Cheers, Mark

ColdFusion per vhosts mappings

Wed, 06 May 2009 13:32:00 --1000

One of the features of CF8 that I was most interested in and most let down on was per application mappings. The reason I was so keen was so that we could keep core library application files out of the webroot for additional security, but still have different versions of the same codebase. I.e. some sites running farcry3 some running farcry5 for example. This didn't transpire and so now we use the following technique on apache to map the folder in and then lock it down with apache. In the vhost section add the following:


Alias /farcry /path/to/non-webfolder/farcry

    Order Deny,Allow
     Deny from all

Note: it is important that there is no /farcry mapping defined in the CF Administrator or it will override this setting. What the above does, is to map in the non-web accessible folder, but then locking it down so it cannot be browsed from the web. This seems pointless but CF can still see the files and correctly resolves the paths even though they are secure from the outside world. This then allows us to version our sites independently - i.e. if we want to upgrade from farcry 5 to 5.1 we can do it one site at a time, instead of having to test them all at once. Note - Railo already has per vhost mappings which solve the same problem as this. But if you are on CF7 or CF8 then this works a treat. Cheers, Mark

ExtendoText project on google code

Wed, 08 Apr 2009 09:09:00 --1000

We've been busy squirrelling away on many projects and they are beginning to come to light: One which is coming to light very early is a new text framework for actionscript which leverages flash 10 text engine. It's very early alpha but is a good proof of concept and already solves some problems for us - namely the ability to simply create rich text area's that have superscript and subscript text. It uses XML and CSS and completely seperates the formatting from the content. It can now:

Render a simple XML document
Apply styles including color, font and opacity
Plug in custom code for rendering images, video or any other XML tag you want to create.

The screenshot highlights a number of these features:

Bold, italic, super and subscript text
Text wrapping around custom display classes (Red and Green boxes)
Opacity applied to display objects (green box)

There is still a lot more to be done - including editing of text which is the next hard piece to knock over. You can get it from the google code site Keen to hear any feedback. Cheers, Mark

HOWTO generate diff files from subversion

Tue, 03 Mar 2009 11:46:00 --1000

Here's a nice and quick way to generate a diff file from subversion and store it on your local machine.


svn diff filepath/myFile.ext > saveDirPath/myFile.ext.diff

Where filePath is the local path to your working directory in which your file is located. And saveDirPath is the path where you want to save your diff file. Diff files can be very useful when contributing to open source projects. You can also get a list of files changed between revisions. Marko