DevGuy

Java :: Inside the firewall RPC choices

Author: devguy
Subject: Inside the firewall RPC choices
Posted: Wed May 07, 2008 5:15 pm (GMT 0)
Topic Replies: 0

Recently the Spring folks released a POJO-based application server that doesn't require J2EE. I'm not sure however if it addresses the case where two out of process objects inside the firewall need to communicate.

How best to communicate between services inside the firewall?

If you use RMI, you need to understand the problems with it. Newton addresses them. If there were no problems Newton and Jini would probably not exist. Try this for starters: http://forum.java.sun.com/thread.jspa?threadID=5235040&tstart=330

RMI has an achilles heel: the ping heartbeat that firewalls can interfere with. Jini technology attempts to work around this via JERI - see http://www.artima.com/intv/jeri.html.

If you don't use RMI, there the following traditional choices: sockets, IIOP, web services (which require load balancing eventually), and JMS.

If you're not going to use RMI, there is little reason to use IIOP, except for interoperability. IMO, avoid IIOP if you can due to its clunkiness.

http overhead is usually unwarranted inside the firewall. The use of http has a typical (although avoidable) side-effect of converting input data to a URI-encoded string and converting output data from XML to objects. Both schemes are platform and language neutral but they are CPU intensive.

Like RMI, sockets are fast - but are not for novices. JMS can scale worse than http because each http server essentially has its own - albeit limited - queue.

JMS has the unique advantage that "RPC" calls can survive either the client or server crashing because clients and servers connect to the JMS server and not to each other. It also makes configuration easier. JMS is also useful because you can poll the server to see how much work is queued up for processing.

Newton doesn't do JMS out of the box. That is where Infiniflow, the not-free version of Newton, comes in apparently. I think that we could get a lot more flexibility out of Infiniflow than we do with Mule. But if there is no Infiniflow, the choices are between Spring JMS, EJB, or Mule. EJB is best left alone. We use Spring JMS today but I would not use it again if I could avoid it.

OpenTerracotta seems to bring most of the best features of RMI and JMS to the table without using either. See:
http://terris.com/tiki/tiki-index.php?page=OpenTerracotta

Unlike http and RMI, JMS introduces client-side lib/jar dependencies that can be a headache. Using JMS behooves you to answer the following vexing questions:

# Can clients receive other client's messages (which are essentially responses to previous requests) or should each client create its own temporary reply-to queue?
# Can a single JMS server handle your needs or do you need JMS clustering?
# What is the fail-over strategy when a JMS server crashes?
# What is the latency of a single JMS server? A JMS cluster? Can you tolerate it? Consider requests and responses.
# What is the message format? Binary of a serialized Java object? XML? JSON?
# Should messages be persistent?
** This applies to the scenario where either a consumer or a producer crashes
** It also applies if there could so many pending messages that they do not fit in memory
# What should you use as your persistent store?
** mysql seems to be better than Derby and Postgres
# Is there a maximum queue size?
# Should one message be received multiple times by different subscribers?
# What is the TTL (time to live) for the message?
# What is the resend timeout for unacked messages?
# What is the priority of the message?
# How many messages should each queue keep in memory?
# What is the needed throughput for the JMS server?
# Should clients manually "ack" messages?
# Can you slow down producers when consumers can't keep up?

Java :: Current date/time as iso8601 string (almost)

Author: devguy
Subject: Current date/time as iso8601 string (almost)
Posted: Fri Apr 25, 2008 10:39 pm (GMT 0)
Topic Replies: 0

Code:

import java.util.Date;
import java.text.DateFormat;
import java.text.SimpleDateFormat;

public class Timestamp
{
/**
* @return The current time in the format yyyy-MM-dd-HH:mm:ss.S
*/
private static final String defaultDateTimeFormat = "yyyy-MM-dd-HH:mm:ss.S";

public static String now()
{
return now(defaultDateTimeFormat);
}

public static String now(String format)
{
DateFormat dateFormat = new SimpleDateFormat(format);
Date date = new Date();
return dateFormat.format(date);
}

public static String nowEpoch()
{
Date date = new Date();
return Long.toString(date.getTime());
}
}

Nagios :: NRPE and check_disk

Author: devguy
Subject: NRPE and check_disk
Posted: Sat Apr 19, 2008 6:02 pm (GMT 0)
Topic Replies: 0

This is surprisingly difficult to figure out.

Warning when <= 20% free space
Critical when <= 10% free space

ARG1 is the path to inspect.

Code:

command_line $USER1$/check_nrpe -H $HOSTADDRESS$ -c check_disk -a "20% --critical=10% $ARG1$"

Java :: RE: What is an Enterprise Service Bus (ESB)?

Author: devguy
Posted: Wed Apr 09, 2008 4:04 pm (GMT 0)
Topic Replies: 1

See also -

http://devguy.com/bb/viewtopic.php?p=3586

Java :: What is an Enterprise Service Bus (ESB)?

Author: devguy
Subject: What is an Enterprise Service Bus (ESB)?
Posted: Wed Apr 09, 2008 4:00 pm (GMT 0)
Topic Replies: 1

A web services company uses JMS in two ways.

In the first approach, Message Driven Beans implement business functionality. Message Driven Beans, by virtue of the obligatory J2EE server, result in a heavyweight and inflexible architecture. Teams can't easily swap in a better messaging transport - JMS is the only option. However, Message Driven Beans effectively hide JMS from the code, making development simpler and permitting the JMS implementation to be replaced.

As an alternative approach, stand-alone Spring daemon console applications communicate via JMS. This approach, while avoiding J2EE and using POJOs, also results in a heavyweight solution.

Firstly, these POJOs tend to be resource hogs because they run on dedicated Java VMs.

Secondly, since java object serialization does not apply to method calls, developers are left to their own devices. Stand-alone JMS applications contain a surprising amount of code implementing start/stop, singleton management, and serialization. Although Spring eliminates much of the JMS-related code, Spring's functionality falls short of that of Message Driven Beans.

Leveraging Message Driven Beans results in a superior solution compared to writing stand-alone applications. Message Driven Beans hide more complexity from developers and a single Java VM can accommodate a broader range of business components.

Is there a way to realize the benefits of Message Driven Beans without resorting to J2EE?

There is. Enterprise Service Bus (Enterprise Service Bus (ESB) technology brings the convenience of Message Driven Beans to the world of lightweight, POJO-centric development.

The Enterprise Service Bus allows POJOs to communicate in a protocol-neutral manner. Unfortunately, various ESB technologies achieve this feat in vastly different ways. Java Business Integration (JBI) seeks to standardize the Java ESB market. ServiceMix, which is still incubating (which doesn't necessarily imply immaturity) is taking the JSR road, having had the luxury to start from scratch with JSR 208 - JBI. Mule and Newton have been more pragmatic and seem to integrate well with JBI containers.

The arguments for and against ESB and JBI are similar to those for JMS, which is an API specification, and AMQP, which is a protocol specification. The JMS camp claims that a protocol specification limits innovation in both features and performance. On the other hand, AMQP promotes interoperability, both within and without the Java universe.

Comparisons between various ESB containers can be found on the Internet, such as here. Mule, ServiceMix, and Newton, all promote POJO-centric, Spring beans development. Like Tomcat, ESB containers provide a runtime environment for POJOs, but without requiring HTTP-based invocation. Reusable Spring beans can run in any sort of ESB container and they can be wired together using RMI, HTTP, JMS, AtomPub, XMPP, and virtually any other protocol.

RMI requires sending client proxy jars for marshalling. Standard java class loaders don't do this robustly when the interface changes. Newton solves this problem with Jini. No other technology we're aware of has solved this problem so completely. Newton's out of the box OSGi support is also impressive. Mule, for example, will not ship with comprehensive OSGi support until at least Q1 2009. Per Mule4Newton's getting started guide:

Quote:

The reason why we're running Mule on Newton is because Newton is designed for distributing applications in a self-healing adaptive network and this gives us failover and scale-out.

On the other hand, Mule and MuleForge provide an impressive amount of connectors and other useful components that are handy even outside the context ESB. A reasonable plan may be to start with Mule and migrate to Newton if need be.

Linux and BSD :: Network Time Synchronization

Author: devguy
Subject: Network Time Synchronization
Posted: Fri Apr 04, 2008 10:34 pm (GMT 0)
Topic Replies: 0

/usr/sbin/ntpdate pool.ntp.org

Java :: Navigating the Jungle of Java Development and Deployment

Author: devguy
Subject: Navigating the Jungle of Java Development and Deployment
Posted: Thu Apr 03, 2008 10:13 pm (GMT 0)
Topic Replies: 0

Navigating the Jungles of Application Development and Deployment

DevGuy.com
April 2008

Technologies such as Apache, Mule, Infiniflow/Newton, Tomcat, Apache CXF, Spring, BPEL, BPM, JMS, JavaSpaces, GigaSpaces, OSGi, REST, WS-*, Apache Abdera, etc. are all useful for delivering value to end-users. But how do developers know which technologies to use, and when?

While it is possible to build a website that runs completely inside a single Apache instance, it's impossible to build scalable and fault tolerant applications this way. This is true whether you use PHP, Java, or even RoR. If you try to load balance a website that follows this architectural pattern, the first crash that usually occurs is caused by running out of database connections, unless the database server fails under the load beforehand.

Business logic needs to reside on a separate tier from that of the web server. This provides greater control of resources and also enhances the security of the system. For example, if hackers break into a web server, they will then need to compromise a business logic server in order to steal, say, database passwords.

Tomcat, Apache, and BPEL servers are useful for building http-based services. However, http's overhead makes it inappropriate for many internal, inside-the-firewall service needs. So where should the business logic tier reside? Should developers build their own Java-based daemon processes that are started in /etc/init.d ?

Probably not. Let's start from the beginning. While building scalable software, the following inverted layer cake often emerges:

1. Business objects are implemented as Plain old Java Objects (POJOs)
2. A runtime environment responds to events, invoking business objects in order to implement potentially complex workflows
3. Magic pixie dust is sprinkled liberally for scalability and fault-tolerance (aka performance)
4. Cold, warm, and hot methods of deployment distribute services to the network
5. Self-regulation senses tension and slack, deploys additional services to available resources automatically, and retires underused services

The depth of the stack varies depending on the project. We have gotten this far without hot deployment and self-regulation, for example.

Development teams are free to choose the best of breed offering at each layer, be they freeware, open source, commercial, etc..

Developers don't necessarily need Spring to build POJOs, but they probably should.

Mule, Newton, ServiceMix, and other players compete in the second layer. Read more about Mule and Spring.

MuleSource and other software vendors seem to believe that SOA is where the money is. Their marketing messages may confuse and deflect potential new users. SOA doesn't necessarily imply that services are invoked via http, but this common assumption may turn off decision makers.

MuleSource is well known, not because of its marketing efforts, but instead because Mule satisfied a diverse set of use cases via a free offering. Paremus, on the other hand, is not receiving an overwhelming amount of attention (have you heard of Infiniflow?) despite their recent foray into dual licensing.

MuleSource's SOA capabilities are useful for inside-the-firewall services, especially when considering Galaxy's registry-based deployment (netboot). Outside-the-firewall SOA is a natural offshoot of Mule's underlying capabilities. Mule touts an excellent WS- and RESTful web services story. With no code changes, POJOs invoked inside the firewall via RMI can be just as easily accessed beyond the firewall via Atom Publishing Protocol clients.

For the third layer, Ambrosia MQ or GigaSpaces might work better than JBOSS Messaging in certain use cases.

On layer #4, there are currently three viable OSGi containers to choose from. And, of course, there are the two deployment powerhouses: Ant and Maven.

For the fifth layer, GigaSpaces and Paremus seem to be the only players. GigaSpaces market themselves as the magic dust that makes Spring and Mule applications perform. This is essentially the same message as that of the various JMS and AMQP vendors. However, in order to use GigaSpaces as a messaging platform, you must first agree to having less than 100% message delivery and secondly, guarantee that every unprocessed message can fit into the sum total of all available RAM across the cluster. These are both high-risk bets in our opinion.

Paremus seems to be more focused on self-healing deployment compared to GigaSpaces, which is likely to result in a better product and a lower price point for Paremus' commercial Infiniflow offering. Read how Newton and Mule work together.

Here are the horses that we're currently betting on:

In layer #1 there's Spring, Hibernate (or a lightweight data access layer), and the usual suspects (log4J, JAXB, etc.)
In layer #2 there are Apache Wicket and Mule
In layer #3 there's JBOSS Messaging
In layer #4 there are Knopflerfish and Maven, facilitated by Newton (Mule will work with OSGi in 2009)
In layer #5 we have Newton

Perl :: Remove whitespace and C style comments from string

Author: devguy
Subject: Remove whitespace and C style comments from string
Posted: Mon Mar 31, 2008 11:08 pm (GMT 0)
Topic Replies: 0

This takes a string of C or SQL code as input and outputs the string without any newline characters or comments.

Quote:

int main(); /* main
main
*/
{
return

foo();
}

and returns:

Quote:

int main();{return foo();}

Code:

sub strip
{
my $val = $_[0];
my @lines = split(/\n/, $val);

foreach(@lines)
{
s/^\s+|\s+$//g;
}

$val = join("", @lines);

$val =~ s{
/\*.*?\*/ ## /* ... */ comment
| ## OR various things which aren't comments:

(
" ## Start of " ... " string
(
\\. ## Escaped char
| ## OR
[^"\\] ## Non "\
)*
" ## End of " ... " string

| ## OR

' ## Start of ' ... ' string
(
\\. ## Escaped char
| ## OR
[^'\\] ## Non '\
)*
' ## End of ' ... ' string

| ## OR

. ## Anything other char
[^/"'\\]* ## Chars which doesn't start a comment, string or escape
)
}{defined $1 ? $1 : ""}gxse;

$val;
}

MySQL :: Fix Can't connect to local MySQL server through socket '/t..

Author: devguy
Subject: Fix Can't connect to local MySQL server through socket '/t..
Posted: Wed Mar 19, 2008 10:39 pm (GMT 0)
Topic Replies: 0

When running 'mysql' on the same computer where the mysql server is running, you get the error:

Code:

Can't connect to local MySQL server through socket '/tmp/mysql.sock'

First, locate the mysql.sock file as root:

Code:

find / -name mysql.sock -print

As root, modify /etc/my.cnf and add:

Code:

[client]
socket=/var/lib/mysql/mysql.sock

The actual value of socket= should contain, of course, the full path where you actually found the file.

MySQL :: Patching DST Daylight Savings Time in mySQL on UNIX

Author: devguy
Subject: Patching DST Daylight Savings Time in mySQL on UNIX
Posted: Wed Mar 19, 2008 10:14 pm (GMT 0)
Topic Replies: 0

First, make sure your operating system is patched with up to date timezone files.

Then run:

Code:

mysql_tzinfo_to_sql /usr/share/zoneinfo | mysql -u root mysql -p

And restart the mysql server

Linux and BSD :: Compensating for the US March 2007 DST Changes

Author: devguy
Subject: Compensating for the US March 2007 DST Changes
Posted: Wed Mar 12, 2008 5:19 pm (GMT 0)
Topic Replies: 0

As root..

yum update tzdata

Then run timeconfig and select a city near you.

Windows :: What to do when changing IIS server's IP address

Author: devguy
Subject: What to do when changing IIS server's IP address
Posted: Tue Mar 11, 2008 3:17 pm (GMT 0)
Topic Replies: 0

1. Run IIS admin
2. Right click on the default website
3. Click on the Web Site tab
4. Click on the advanced button
5. Change the IP addresses. The bottom section is for https only - that has confused me many times.

Perl :: Beginnings of logical model to PostgreSQL Schema

Author: devguy
Subject: Beginnings of logical model to PostgreSQL Schema
Posted: Sat Mar 08, 2008 3:25 am (GMT 0)
Topic Replies: 0

Code:

#! /usr/bin/perl

##### User-specified Logical Schemas #####

##### Enterprise Reporting March 2008 #####
#
package mm::er::model::2008::03;

sub new
{
$model = bless{'name'=>'MarkMonitor Enterprise Reporting Schema',
   'version'=>'2008-March'},shift;

$model{'modeler'} = shift;
$model;
}

sub build
{
my ($this) = @_;
$this->addAuditEntity();
$this->addCustomerTable();
}

sub addAuditEntity
{
my ($this) = @_;
my $entity = $this->{'modeler'}->addEntity('audit');

$$entity{'columns'} =
[
   {
      'name'=>'_added',
      'type'=>[DT],
   },
   {
      'name'=>'_modified',
      'type'=>[DT],
   },
];

0;
}

if (0)
{
sub addBigEntity
{
$modeler->addEntity('big');

$$_{'prefix'} = "_";

$$_{'columns'} =
[
   {
      'name'=>'added',
      'type'=>[DT],
   },
   {
      'name'=>'modified',
      'type'=>[DT],
   },
];

0;
}
}

sub addCustomerTable
{
my ($this) = @_;
my $table = $$this{'modeler'}->addTable('customer');

$$table{'inherit'} =
[
   {
      'entity'=>'added',
      'type'=>[DT],
   },
   {
      'name'=>'modified',
      'type'=>[DT],
   },
];
}

##### Engine Logic #####
package dg::modeler::2008::03;

%types = (DT=>{sql=>"time with timestamp"},
INT32=>{sql=>"int"},
UINT32=>{sql=>"int"},
INT64=>{sql=>"bigint"},
TEXT=>{sql=>"text"},
STRING=>{sql=>"varchar(%s)"},
SERIAL=>{sql=>"serial",reftype=>UINT32},
BIGSERIAL=>{sql=>"serial",reftype=>INT64},
   );

sub new
{
bless{'name'=>'DevGuy Modeler',version=>'2008-03-20'},shift;
}

# semipublic
sub setModel
{
shift->{'model'} = shift;
0;
}

# public
sub process
{
my ($this, $model) = @_;
$this->{'model'} = $model;
$model->{'modeler'} = $this;
$model->build();
0;
}

# public
sub destroy
{
undef shift->{'model'};
}

# semipublic
sub addEntity
{
my ($this, $name) = @_;
$$this{'model'}->{'entities'}->{$name} = {'name'=>$name};
}

# semipublic
sub addTable
{
my ($this, $name) = @_;
$$this{'model'}->{'tables'}->{$name} = {'name'=>$name};
}

sub getTypeSQL
{
my $tn = $types{pop @_};
my $result = printf($tn, @_);
die "Unknown type $tn" unless $result;
$result;
}

sub camelToUnderscore
{
my ($str) = @_;
my $ret = lc substr($str,0,1);
$str = substr($str,1);
$str =~ s/([A-Z])/_\1/g;
lc $str;
}

sub columnName
{
my ($column) = @_;
my $prefix = $$column{'prefix'};
my $name = camelToUnderscore($column->{'name'});
}

sub processTable
{
#my ($this, $table) = @_;
#%pTable{'name'} = camelToUnderscore($$table{'name'}) . '_t';
#$tables{
}

sub processSchema
{
#foreach(@$tables)
#{
# processTable($_);
#}
}

sub createColumn
{
my ($column) = @_;
my $sql = sprintf("create table '%s' (\n", $$table{'name'});
}

sub createTable
{
my ($table) = @_;

my $sql = sprintf("create table '%s' (\n", $$table{'name'});
my $columns = $$table{'columns'};

my $first = 1;
foreach ($$table{'columns'})
{
   if ($first)
   {
      $first = 0;
   }
   else
   {
      $sql .= ",\n";
   }

   createColumn($_);
}
$sql .= "\n)";
0;
}

sub generateDDL
{
my ($schema) = @_;
processSchema($schema);

my $str;
open(F, ">", \$str);

open(SAVEOUT, ">&STDOUT");
open(SAVEERR, ">&STDERR");
open(STDOUT, ">&F") || die "Can't redirect stdout";

eval
{
open(STDERR, ">&STDOUT") || die "Can't redirect err";
foreach($$schema{'tables'})
{
      createTable();
}
};

my $err = $@;

open(STDOUT, ">&SAVEOUT");
open(STDERR, ">&SAVEERR");

die $err if $err;

$str;
}

##### MAIN #####
package main;

sub main
{
my $modeler = new dg::modeler::2008::03;

{
   $modeler->process(mm::er::model::2008::03->new);
   my $ddl;
   #$modeler->appendDDL($ddl);
   #print $ddl;
}
}

main();