Lynch Consulting Blog - ColdFusion

ServerStats mentioned on CFHour

Tue, 29 Jun 2010 13:34:00 --1000

Just came across a mention of a little project that I did back in 2006 in the most recent CFHour podcast - Show #58 - Monitoring, Debugging, and Guests. (Mention is at approx 53 minutes). The ServerStats code wraps up a few undocumented methods for CF to make it easy to get information on the number of sessions active on a server as well as some basic information on memory usage. Thanks to Charlie Arehart for mentioning it - keep up the good work. I must have a look at updating it to support Railo when I get a moment. Cheers, Mark

proxy_http vs proxy_ajp benchmark

Thu, 18 Mar 2010 12:20:00 --1000

After I posted a previous blog entry about configuring railo & tomcat with apache and mod_proxy_http, Paul Kukiel and Gary Gilbert suggested that I should be using mod_proxy_ajp. This has been something I've been looking at, but haven't found a compelling reason for one over the other. Proxy AJP is claimed to be faster as it is a "Wire protocol" but I couldn't find any benchmarks around this. So I decided to do a very quick and dirty benchmark to satisfy my curiosity. This is not a scientific process, I just ran a simple railo testpage on the same machine with 50 threads of jmeter requests hitting it. First I enabled proxy_http and ran it four times, then enabled proxy_ajp and repeated. The config is below:


# Proxy HTTP config

	
	Order deny,allow
	Allow from all
	
	ProxyPassMatch ^/(.*\.cfm)$ http://testsite.railo:8080/$1
	ProxyPassReverse  /  http://testsite.railo:8080/


# Proxy AJP config

	
	Order deny,allow
	Allow from all
	
	ProxyPassMatch ^/(.*\.cfm)$ ajp://testsite.railo:8009/$1
	ProxyPassReverse  /  ajp://testsite.railo:8009/

Results:

Run	HTTP Requests/sec	AJP Requests/sec
Run 1	206.9	181.4
Run 2	203.9	143.6
Run 3	194.6	189.2
Run 4	204.6	191.4
Average	202.5	176.4

The results showed that the proxy_http module was faster - i.e. more requests per second could be pushed through. I'm putting this down to the fact that proxy_ajp has to convert the http request into it's binary format, while proxy_http really just has to pass it along. In different scenarios and network configurations the results may be different, but for now I'm going to stick with the http proxy. Proxy AJP has one other benefit, in that is passes along some extra flags such as whether the request is https or not, but for our purposes we don't need this. Cheers, Mark

Railo on Tomcat revisited - mod_proxy

Fri, 12 Mar 2010 11:01:00 --1000

Updated: Changed the linking between railo and tomcat to use shared.loader. I've been doing some more work on configuring railo to work flexibly in the numerous different environments we work in, and also making it simpler to set up. To that end I investigated the use of mod_proxy for linking it to apache instead of mod_jk. Advantages of this approach are:

Simple - communications are in plain http
Flexible - Load balancing can be easily added at the apache layer
Simple - No compiling mod_jk

Here are the basic install instructions for Railo/Tomcat/Apache on Ubuntu.

Download & Install Tomcat

Download tomcat and extract content:


tar xvzf apache-tomcat-6.0.26.tar.gz

Move Tomcat to a more appropriate place:


sudo mv apache-tomcat-6.0.26 /opt/tomcat

Download Railo

Download Railo custom version jars file Extract and move into Tomcat lib directory:


tar zxvf railo-3.1.2.001-jars.tar.gz
sudo mv railo-3.1.2.001-jars /opt/railo

Make Tomcat load the railo jars by editing catalina.properties to change the shared loader path:


shared.loader=/opt/railo/*.jar

Make Tomcat and Railo work together by modifying the web config file:


sudo nano -w /opt/tomcat/conf/web.xml

add the following inside the <web-app> element:



CFMLServlet
railo.loader.servlet.CFMLServlet
   
      configuration
      {web-root-directory}/WEB-INF/railo/
      Configuraton directory
   
   1


   CFMLServlet
   *.cfm


   CFMLServlet
   *.cfml


   CFMLServlet
   *.cfc

add the following inside <welcome-file-list> element:


index.cfm
index.cfml

Start up tomcat:


/opt/tomcat/bin/startup.sh

Once this is done you should be able to access the railo admin by going to the following URL:


http://127.0.0.1:8080/railo-context/admin/server.cfm

Back to Tomcat

To test our Railo installation, let's create a test site by adding a new virtual host in both Tomcat and Apache. We do this by modifying Tomcat server.xml file (/opt/tomcat/conf/server.xml )

Linking with Apache via Mod Proxy

Ensure the modules proxy and proxy_http are enabled. On Ubuntu this is done as follows:


sudo a2enmod proxy
sudo a2enmod proxy_http
sudo /etc/init.d/apache2 restart

Create vhost

Now we need to create a virtual host entry in Apache as well:



        DocumentRoot /vhosts/testsite.railo/www
        ServerName testsite.railo
        DirectoryIndex index.cfm
	#Proxy .cfm requests to railo
	
		
		Order deny,allow
		Allow from all
		
		ProxyPassMatch ^/(.*\.cfm)$ http://testsite.railo:8080/$1
		ProxyPassReverse  /  http://testsite.railo:8080/
	

	#Deny access to admin except for local/portforwarded clients
	
		Order deny,allow
		Deny from all
		Allow from 127.0.0.1

This tells apache to forward all requests for CFM files to the railo instance. Finally restart apache and railo and you should be good to go.


sudo /opt/tomcat/bin/shutdown.sh 
sudo /opt/tomcat/bin/startup.sh 
sudo /etc/init.d/apache2 restart

Upcoming book review - Tomcat 6 Developer's Guide

Mon, 21 Dec 2009 21:12:00 --1000

I've just received a copy of the Tomcat 6 Developer's Guide from packt publishing to review. It's nice timing as I've been working with Tomcat 6 a bit lately and in the new year plan to move some of our production systems over to running Railo on top of Tomcat. After the extremely busy year Learnosity has had I'm looking forward to reading a few books over the break and coming back in the New Year with lots more ideas and technology to implement.

Learnosity are looking for a Web Ninja at a Mid to Senior level

Fri, 04 Dec 2009 16:03:00 --1000

4 December 2009, Learnosity are looking for a Web Ninja at a Mid to Senior level.

About Learnosity

Learnosity develop cutting edge tools for teachers and educators. Our flagship product Learnosity Voice uses the telephone to enable language students and teachers to interact on a one to one level. Our service:

Makes it practical for students to practice Oral and Aural skills
Is efficient and effective for teachers, as they can listen to each student individually at a time to suit them
Can be used for homework assignments or “High Stakes Assessments”

We are continuing to grow our core development team and we need another great developer to help us keep up with demand.

We need someone who can:

Use Javascript or Actionscript to create great user interfaces
Develop highly scalable web applications
Cut code with the best in the world

You will also need to be:

energetic with a butt kicking attitude
ready to create cutting edge web 2.0 apps
keen to continue learning new technologies
able to have a conversation with non technical people

You'll need:

3 or more years of programming experience
Expert in at least on Client side language (Actionscript or AJAX)
Expert in at least one Server side language (eg PHP, Java, ColdFusion, etc)
Understanding of Object Oriented design
Understanding of XHTML and CSS

It would be good if you have:

A degree in Computer Science, Engineering or similar.
been working with open source tools
been playing around with iPhone/Android applications
experience with some of Linux/VOIP/SIP/Asterisk/Jabber/XMPP

This is a full time role and you will be working in a casual workplace with flexible hours in the Sydney CBD. If this sounds like the job for you, email a covering letter explaining why you'll be great and your resume to mark@learnosity.com - no agencies please.

Chinese characters not working cfdocument for PDF exports - fixed

Mon, 26 Oct 2009 16:38:00 --1000

I've just spent quite a few hours grappling with an annoying PDF export issue where the font's were not displaying correctly. This is on a CFMX 7 (7.0.2) on Ubuntu Linux system. All the DB storage and application was using UTF8 throughout, but the PDF exports were showing nothing when they should have been showing Chinese characters. To fix this up you need to do the following:

1. Need fonts installed on system:


sudo apt-get install ttf-arphic-bsmi00lp ttf-arphic-gbsn00lp

2. Need fonts installed in CFAdmin

Go to CFAdmin and Font Management
Select directory /usr/share/fonts/truetype/arphic
Select "Add" and you should see 2 new fonts added.

3: CFdocument Hotfix for CF7.0.2

The CFDocument hotfix for 7.0.2 provides some essential fixes and makes this work - Follow the instructions to install: http://kb2.adobe.com/cps/402/kb402093.html

Configure cffont.properties file

The final step is to set the cffont.properties file so that it will check additional font sets for any characters it doesn't know about - i.e. Chinese characters. Each line in the file is like a CSS font-family rule, i.e. it will start at the left and look for the correct font to render the text. I appended the two extra font names to ensure they are checked, so the file looks like this:


defaultbasefont=
dialog=Arial, Helvetica, AR PL SungtiL GB,AR PL Mingti2L Big5
dialog.bold=Arial Bold, Helvetica-Bold, AR PL SungtiL GB,AR PL Mingti2L Big5
dialog.italic=Arial Italic, Helvetica-Oblique, AR PL SungtiL GB, AR PL Mingti2L Big5
dialog.bolditalic=Arial Bold Italic, Helvetica-BoldOblique, AR PL SungtiL GB, AR PL Mingti2L Big5
dialoginput=Courier New, Courier, AR PL SungtiL GB, AR PL Mingti2L Big5
dialoginput.bold=Courier New Bold, Courier-Bold, AR PL SungtiL GB, AR PL Mingti2L Big5
dialoginput.italic=Courier New Italic, Courier-Oblique, AR PL SungtiL GB, AR PL Mingti2L Big5
dialoginput.bolditalic=Courier New Bold Italic, Courier-BoldOblique, AR PL SungtiL GB, AR PL Mingti2L Big5
serif=Times New Roman, Times-Roman, AR PL SungtiL GB, AR PL Mingti2L Big5
serif.bold=Times New Roman Bold, Times-Bold, AR PL SungtiL GB, AR PL Mingti2L Big5
serif.italic=Times New Roman Italic, Times-Italic, AR PL SungtiL GB, AR PL Mingti2L Big5
serif.bolditalic=Times New Roman Bold Italic, Times-BoldItalic, AR PL SungtiL GB, AR PL Mingti2L Big5
sansserif=Arial, Helvetica, AR PL SungtiL GB, AR PL Mingti2L Big5
sansserif.bold=Arial Bold, Helvetica-Bold, AR PL SungtiL GB, AR PL Mingti2L Big5
sansserif.italic=Arial Italic, Helvetica-Oblique, AR PL SungtiL GB, AR PL Mingti2L Big5
sansserif.bolditalic=Arial Bold Italic, Helvetica-BoldOblique, AR PL SungtiL GB, AR PL Mingti2L Big5
monospaced=Courier New, Courier, AR PL SungtiL GB, AR PL Mingti2L Big5
monospaced.bold=Courier New Bold, Courier-Bold, AR PL SungtiL GB, AR PL Mingti2L Big5
monospaced.italic=Courier New Italic, Courier-Oblique, AR PL SungtiL GB, AR PL Mingti2L Big5
monospaced.bolditalic=Courier New Bold Italic, Courier-BoldOblique, AR PL SungtiL GB, AR PL Mingti2L Big5

The cffont.properties file is located in /opt/jrun4/servers/{instance name}/cfusion.ear/cfusion.war/WEB-INF/cfusion/lib on the jrun multiserver version.

5. Restart CF

Once you restart CF you documents should be coming out with their Chinese fonts intact. I'm sure there are additional fonts to add to get it working natively with all the other asian languages, so I'll add to this as and when I need/discover them. Please feel free to post a comment if you find any more font's that should be added to the list.

CFMX Query Timeout Gotcha

Tue, 19 May 2009 14:04:00 --1000

NOTE: I found this blog post in my to be posted ones from 2008. While doing some load testing a while back we came across a strange error while simulating the load that we were expecting. The simulated load had a large first peak, followed by a quieter period, followed by some more peaks. During the load test we noticed a few query errors that we had never seen before, they related to the query connection timing out. It turns out that the way CF times out datasource connections is slightly quirky as described below:

The other timeout setting in the CF Admin datasource detail page is just labeled as Timeout, with a paired setting for Interval. This timeout value is how long a connection should be idle before it is removed. So if a connection goes unused for this period of time, then the next time ColdFusion checks that connection it will be removed from the pool. The frequency that ColdFusion checks the connection idle time is set by the Interval setting. The default for the Timeout setting is 20 minutes and the Interval is 7 minutes, so every 7 minutes ColdFusion will check connections for that datasource and remove connections that have been idle for 20 minutes or more. The number of connections idle connections removed from the pool at each interval is hardcoded in ColdFusion to just five. Say you have 20 idle connections hanging around in a pool, this would require at least 28 minutes to be cleaned up.

Timeout (min)

The number of minutes that ColdFusion MX maintains an unused connection before destroying it.

Interval (min)

The time (in minutes) that the server waits between cycles to check for expired data source connections to close.

Cheers, Mark

ColdFusion per vhosts mappings

Wed, 06 May 2009 13:32:00 --1000

One of the features of CF8 that I was most interested in and most let down on was per application mappings. The reason I was so keen was so that we could keep core library application files out of the webroot for additional security, but still have different versions of the same codebase. I.e. some sites running farcry3 some running farcry5 for example. This didn't transpire and so now we use the following technique on apache to map the folder in and then lock it down with apache. In the vhost section add the following:


Alias /farcry /path/to/non-webfolder/farcry

    Order Deny,Allow
     Deny from all

Note: it is important that there is no /farcry mapping defined in the CF Administrator or it will override this setting. What the above does, is to map in the non-web accessible folder, but then locking it down so it cannot be browsed from the web. This seems pointless but CF can still see the files and correctly resolves the paths even though they are secure from the outside world. This then allows us to version our sites independently - i.e. if we want to upgrade from farcry 5 to 5.1 we can do it one site at a time, instead of having to test them all at once. Note - Railo already has per vhost mappings which solve the same problem as this. But if you are on CF7 or CF8 then this works a treat. Cheers, Mark

Railo Useful links

Tue, 24 Feb 2009 12:30:00 --1000

I've been doing some research on Railo with a view to moving some of our sites to using it. I've found a few useful links along the way:

My evaluations of Railo are so far so good, with performance being one of the key issues and for our application I've seen performance 30% faster than CF7 and 25% faster than CF8. If I find any major issues or benefits I'll post them here. Cheers, Mark

Identifying which queries to tune with MSSQL

Wed, 04 Feb 2009 11:29:00 --1000

I've been doing some more work tuning and CF/MSSQL application and during figuring out which parts of the system is doing the most work (and hence a good candidate for tuning) I came across this gem which will show stats on the most run queries in the cache.


SELECT TOP 20 SUBSTRING(qt.text, (qs.statement_start_offset/2)+1, 
        ((CASE qs.statement_end_offset
          WHEN -1 THEN DATALENGTH(qt.text)
         ELSE qs.statement_end_offset
         END - qs.statement_start_offset)/2)+1), 
qs.execution_count, 
qs.total_logical_reads, qs.last_logical_reads,
qs.min_logical_reads, qs.max_logical_reads,
qs.total_elapsed_time, qs.last_elapsed_time,
qs.min_elapsed_time, qs.max_elapsed_time,
qs.last_execution_time,
qp.query_plan
FROM sys.dm_exec_query_stats qs
CROSS APPLY sys.dm_exec_sql_text(qs.sql_handle) qt
CROSS APPLY sys.dm_exec_query_plan(qs.plan_handle) qp
WHERE qt.encrypted=0
ORDER BY qs.total_logical_reads DESC

I found it in this excellent article on optimising MSSQL The times that are returned from the queries are in micro seconds as documented on msdn. Cheers, Mark

Clear MSSQL Query Caches

Tue, 03 Feb 2009 22:12:00 --1000

Found this useful little snippet for when you are load testing MSSQL Servers. You can use it to clear the query cache - to ensure you are starting from a level playing field. Or if you are tuning queries in a application and want to ensure they are not cached run it before each run:


dbcc freeproccache
go
dbcc dropcleanbuffers
go

Cheers, Mark

Non-case sensitive filesystem on Linux - HOWTO

Mon, 02 Feb 2009 20:58:00 --1000

I was setting up a ColdFusion web application on my dev machine and ran into lots of errors because of the case sensitivity of Linux (which i was using) versus the non-case sensitivity of windows (where the application was developed). I had a quick look to see if I could fix it up but quickly figured out that there were too many occurrences and so it was likely not worth fixing up at this point. To that end I set up a loopback vfat filesystem so that I could have a directory on my machine mounted as non-case sensitive. Note: this is not recommended for production systems - but is a handy fix for a temporary development problem.

Create a virtual disk


dd if=/dev/zero of=virtual.dsk bs=1048576 count=150

Format it


mkfs.vfat virtual.dsk

Mount it


sudo mkdir /mnt/vfat
sudo mount virtual.dsk /mnt/vfat -t vfat -o loop,owner,group,umask=000

You can set this up to mount every time by putting the following line in your /etc/fstab


/path/to/virtual.dsk /mnt/vfat  vfat  loop,owner,group,umask=000  0 0

Note - the same trick works on OSX in reverse. I.e. if you are developing on a mac, but deploying to a linux production environment you can create a virtual disk with a case-sensitive filename, which will mean that any case sensitive issues get picked up on your dev machine, and not on your production/staging machines. Cheers, Mark

Remote collection of vmstat log files

Tue, 02 Dec 2008 09:42:00 --1000

We've been doing a lot of load testing recently and are planning on doing a lot more, so we've developed a simple little script to make the collection of the vmstat (performance statistics) a bit quicker. multivmstat is a php command line script which makes it easy to collect the statistics. Download the script To run it you specify a list of servers to check:


./multivmstat server1,server2,server3 5

This will run vmstat on each of the servers with a 5 second interval between samples. It will create the following files:


server1-vmstat.log
server2-vmstat.log
server3-vmstat.log

Note: You mush have ssh access to each machine - ideally with a certificate so that no interactive authentication is required. Download the script

MySQL and Temporary tables and CASE syntax

Sun, 31 Aug 2008 20:47:00 --1000

For some reason I've always avoided the use of temporary tables in SQL (MySQL or otherwise) but today I came across a difficult problem that temporary tables made very simple. I needed to use a case function on a subquery value to return a tri-state value. I.e. yes, no or partial. The cut down query below demonstrates how it works.

First we drop the temporary table in case one has been left lying around.
Then we select a query into the temporary table which include some complex subqueries.
We can then query the temporary table to do further processing on the result, which in this case depending on the values of the class_count and class_count_assigned allows us to set a field in three states.
- 'yes' if all users for a class are selected
- 'partial' if some users are selected
- 'no' if none are selected.



	DROP TEMPORARY TABLE  IF EXISTS tmp_exams;
	/* Create temporary table query which includes student counts */
	CREATE TEMPORARY TABLE tmp_exams
	SELECT	e.id
			, e.name
			, (SELECT count(1) FROM tbl_user_classes uXc
                WHERE uXc.school_class_id = ) AS class_count
            , (SELECT count(1) FROM tbl_user_classes uXc
                INNER JOIN  tbl_users_exams uXe ON uXe.user_id = uXc.user_id
                WHERE uXc.school_class_id = 
                AND exam_id = e.id) AS class_count_assigned
	FROM	tbl_exams e;
	/* Select out the relevant info */
	SELECT e.id
			, e.name
			, e.class_count
			, e.class_count_assigned
		    ,(CASE WHEN class_count_assigned = 0 THEN 'no'
		    	WHEN class_count = class_count_assigned THEN 'yes'
		    	ELSE 'partial' END) AS active
	FROM tmp_exams e
	WHERE	1=1
	ORDER BY e.name

Mostly posted for my future reference but hope it helps out. Cheers, Mark

CF Sandbox Security Tricks and Tips

Wed, 13 Aug 2008 11:07:00 --1000

I've been working on getting CF Sandbox security working. It's trickier than I first thought so here's how to do it for future reference:

Enable security Manager

Instructions from Steven Erat's blog:


Stop ColdFusion.
Locate the jvm.config file in jrun_root/bin.
Back up the file.
Open the file in a text editor.
Add the following lines to the java.args section:
-Djava.security.manager
"-Djava.security.policy=[cf_webapp_root]/WEB-INF/cfusion/lib/coldfusion.policy"
"-Djava.security.auth.policy=[cf_webapp_root]/WEB-INF/cfusion/lib/neo_jaas.policy"

NOTE the example from the adobe site has the quotes in the wrong place. Note you also need to change [cf_webapp_root] to match the location on your machine.

Datasources

After I enabled Datasource security I began to receive the following error, a bit of digging reminded me that the MySQL connector tries to do some autoconfiguration for coldfusion to optimise it.


Message: Can't find configuration template named 'coldFusion' 
Type: java.sql.SQLException

It was failing to load the configuration file which is inside the mysql connector jar file /com/mysql/jdbc/configs/coldFusion.properties I haven't gotten to the bottom of why it couldn't be loaded but adding the following to the datasource query string fixed it up:


autoConfigureForColdFusion=false

I would suggest adding some of the settings from this file as paramaters in your datasource settings as per previous post Note: This was using the following:

ColdFusion 7.0.2 Cumulative Hotfix 1 Multiserver install
MySQL connector/J 5.0.8

Hope it helps. Cheers, Mark