SQL Server is based off Sybase. They both use Transact SQL. However I've found some differences in behavior (at least in the version of Sybase and driver I'm using).
You cannot do a count in a select in sybase without a group by and get accurate results.
COUNT without GROUP BY:
Select count(some_field) from whatever_table will return every row in the table. That select statement works in SQL Server without the group by. In Sybase you'll need to add group by like this:
select count(some_field) from whatever_table group by (some_field)
COLUMNS and GROUP BY:
If you don't have all the columns you are selecting in the group by you may get incorrect results but no error. In SQL this will give you an error like this:
Column 'some_field' is invalid in the select list because it is not contained in either an aggregate function or the GROUP BY clause.
In Sybase something like this will just give you inaccurate results:
select some_field, other_field
from whatever_table
group by other_field
EMPTY STRING, LTRIM, LEN
- insert
If you try to insert an empty string using an insert statement with Sybase it inserts a 1 character space instead of an empty string. Length will equal 1.
insert into some_table (id, some_field) values(23, '')
select len(some_field) from some_table where id = 23 (this will give you length of 1)
In SQL Server the above statements would 1.) insert a true empty string and 2.) return a length of 0.
- ltrim and length
In Sybase ltrim(some_field) above would give you a null and len(ltrim(some_field)) will give you null - no matter how many spaces you have in some_field.
In SQL Server the same scenario would give you an empty string after trim and 0 for the length.
Friday, January 14, 2011
Saturday, December 25, 2010
Finding Memory Leaks in Java Applications
Trying this out to analyze memory usage and find leaks in Java application:
http://blog.emptyway.com/2007/04/02/finding-memory-leaks-in-java-apps/
Pretty slick...
In Eclipse for my debug configuration I put this in the vm arguments box:
-Dcom.sun.management.jmxremote
-Dcom.sun.management.jmxremote.port=9000
-Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote.ssl=false
-agentlib:hprof=heap=dump,file=/tmp/hprof.bin,
format=b,depth=10
A few more details for those not so familiar with Java:
jconsole.exe is found in the /bin directory in the directory where you installed the Java JDK. It is an application with a GUI so just double click on it to open it up. You then select your application from the list to view details about it.
jps.exe is run from command prompt. It is also found in the /bin directory in the directory where you installed the Java JDK. Go to a command prompt (start menu, run, enter "cmd" hit enter) and navigate to that bin folder or alternatively you can type out the full path to jps.exe to run it.
jmap.exe is also in the /bin directory. You also would run that from the command line by typing in the location where you want to write the file and the process id as described in the above article.
jmap -dump:format=b,file=/tmp/java_app-heap.bin 15976
Once that file is created,run jhat as directed.
jhat -J-Xmx326m /tmp/java_app-heap.bin
Then in browser go to:
http://localhost:7000
You can browse that data to find which objects are most in use, how much memory they are taking up, etc.
If a very slow leak can run the app in production using the above to output file over time and then view it.
Seems like support for Eclipse Memory Analyzer and Eclipse TPTP are dwindling but this method above is pretty simple.
http://blog.emptyway.com/2007/04/02/finding-memory-leaks-in-java-apps/
Pretty slick...
In Eclipse for my debug configuration I put this in the vm arguments box:
-Dcom.sun.management.jmxremote
-Dcom.sun.management.jmxremote.port=9000
-Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote.ssl=false
-agentlib:hprof=heap=dump,file=/tmp/hprof.bin,
format=b,depth=10
A few more details for those not so familiar with Java:
jconsole.exe is found in the /bin directory in the directory where you installed the Java JDK. It is an application with a GUI so just double click on it to open it up. You then select your application from the list to view details about it.
jps.exe is run from command prompt. It is also found in the /bin directory in the directory where you installed the Java JDK. Go to a command prompt (start menu, run, enter "cmd" hit enter) and navigate to that bin folder or alternatively you can type out the full path to jps.exe to run it.
jmap.exe is also in the /bin directory. You also would run that from the command line by typing in the location where you want to write the file and the process id as described in the above article.
jmap -dump:format=b,file=/tmp/java_app-heap.bin 15976
Once that file is created,run jhat as directed.
jhat -J-Xmx326m /tmp/java_app-heap.bin
Then in browser go to:
http://localhost:7000
You can browse that data to find which objects are most in use, how much memory they are taking up, etc.
If a very slow leak can run the app in production using the above to output file over time and then view it.
Seems like support for Eclipse Memory Analyzer and Eclipse TPTP are dwindling but this method above is pretty simple.
Sunday, November 21, 2010
Postini, IMail, SSL/TLS
Here's a run down on setting up Postini with IMail. Really quick...disclaimer in case of error but here are my notes...
Postini
You probably all run some kind of mail software with spam blocking capabilities. However the painful thing is that spammers are out there inserting mail directly into your mail server and overloading spam boxes all the time. This takes up resources on your mail server(s) and clients.
What's kind of neat about Postini is that it's a filtering service in front of your mail server. You can force all inbound and outbound mail to go through Postini if you have your own mail server or a service that offers Postini. By setting up Postini correctly a lot of the garbage bombarding your server can be stopped before it ever gets that far.
Postini was bought out by Google and as you can imagine has a tad bit more mail to analyze than the average company hosting an email system. By passing all the garbage through them first, I would hope they can do some pretty good analytics on mail going through their system and figure out how to weed out the junk, the same way they figure out how to put the best web sites at the top of the search engine results [hopefully!]
Postini blocks viruses, spam and has a bunch of cool features like allowing you to block IP ranges from sending mail to your server. Since a lot of spam comes from certain parts of the world, if you don't do business in those areas you can simply block out entire spam friendly networks completely. I found that blocking out everything but ARIN significantly reduced my spam on a particularly old domain that is probably in every spammer's list in existence.
The other cool thing about Postini is that it allows you to force TLS between two parties - for regulatory copmliance or simply because you want to know your email transmission is encrypted end to end (which can be hard to do).
These are just a few of the highlights of Postini service.
The biggest problem with Postini is that, if you're a small business, getting support is challenging. You don't always get answers and when you do they are vague or not actually answering the question you asked or simply don't resolve the problem. Lately, however, I haven't had a ton of questions which is good. The service has been pretty reliable.
IMail
If you're not interested in spending mega-bucks on Exchange, IMail is a simpler, less expensive option and the company provides very good support. When it comes to email that's important to me because I find email to be a royal pain. As noted in a recent episode of American Greed, spam is the backbone of organized Internet crime. They are all trying to get into your in box to get you to click a link - or something.
When people's email isn't working - they want it fixed. Now. Although I'm pretty handy with Java, open source typically comes with no or minimal support. Additionally open source has so many people contributing that are not quite held accountable in the same way I would expect a company with paid staff to be. Mostly, I'd rather pay a bit more and get someone who can help me fix things fast than futz around with user groups and mailing lists or reverse engineering code to find a fix to a type of technology I find completely annoying because it is hard to secure, unreliable and always under attack. I prefer to deal with email as little as possible. I just want to call someone and get an answer. There is one person at Ipswitch in particular I have talked to over the years who is extremely knowledgeable about IMail and was able to quickly answer all my questions during set up.
For me running on IIS was kind of a pain because I prefer Java and it's another thing to manage, but I've used Imail in the past and it's reliable with good support as mentioned. For those who like and know IIS this would be a plus. Setting up the SSL cert was much easier with IIS than Java. The web site runs a little slow but going to look into how that can be tweaked. Using IMail is, for the most part, a piece of cake. The only thing I found challenging (having a basic understanding of mail servers) was getting the SSL cert installed and finding all the parts and pieces to make it work with Postini.
Set Up
--IMAIL--
> Purchase or get trial at IPSwitch.com
> Install - call for help as needed.
> Note that MX records will be pointed at Postini (see below) not your mail server.
--POSTINI--
> Set up an account and login to the System Administrator
> Click on the Orgs & Users tab
> Select your top level account from the drop down list of orgs
> Click the link (name) of your account (top node in the list)
> Enter a new org name in the box at the top and click the "add" button
> Click on orgs & users tab to get the full list of orgs
> Click on the name of the org you just created to view the settings.
> Click on the "general settings" link 3/4 down the page
> Change "email config type org" to Yes and save [allows mail server settings for this org]
> Click on "DNS Instructions" for MX records to use...
--EDIT YOUR DNS--
> edit your MX records to include the specified servers (note that each domain should have the domainname itself in the MX record - replace the part that says yourdomain with the domain for which you are setting up email)
--POSTINI--
> Click on "domains" under the tabs and add a domain
> Click on "users" under the tabs and createa user
> Click on "Inbound Servers" tab, "Delivery Manager", then "Edit" to add your mail servers. If you have your own mail server choose "Use my own mail server" and % to balance load to each server.
> There are other cool features you can look into later...manually block mail from certain IPs (bottom right when you click on "Inbound Servers"), force TLS between certain domains, etc.
--IMAIL--
> In your mail server allow relay from Postini (ONLY so you don't create an open relay for spammers!) In the latest version of Imail you would click on "Services" on the left, then double click on "Imail SMTP service".
> Under "General" click on the "Addresses" tab. Select "Group" from the drop down and enter the Postini IP addresses specified for your outbound mail configuration. The IP addresses depend on which server number you were assigned when you signed up for the service. The server number is in your MX records specified in the DNS instructions. More in this document:
Postini Outbound Services
--POSTINI--
> Click on "Outbound Servers" tab, and at the bottom left choose "Add Another Outbound Email Server" and add your mail server(s) that will be used for sending bounces back to your users.
--IMAIL --
> Set up webmail per the IMail instructions.
> If you want webmail to show up on it's own domain with no directory at the end and no access to the administrator and other things from that domain, set up a second web site on IIS and point it specifically at the webmail folder instead of the parent directory. In other words instead of mail.mydomain.com/web_mail_folder to get to mail you point IIS web site at the web_mail_folder and then can get to that same thing by going to mail.mydomain.com (if that doesn't make sense...call IPSwitch. They are very helpful.)
-- SSL CERTIFICATE (webmail and mail traffic) --
> In IIS create a cert request. I used Digicert.com and they were extremely easy to work with. Click on the IIS link on the left side of their home page and scroll to the bottom to get instructions for IIS cert request.
> When you get back the certificate, install it in IIS (for the domain you're running webmail on).
> Test your webmail domain and make sure it's accessible via SSL (https://mail.mydomain.com)
> For Digicert there were some chained certificates I needed to install in IMail. Originally exporting from IIS didn't work because the output didn't include the full chain. So...
> Use the MMC to export the cert with the full chain of certs:
http://www.digicert.com/ssl-support/pfx-import-export-iis-7.htm
> These "unsupported" instructions will get you two files that have the private key and the cert in separate files - I don't know if there's any issue with this...but I did it and it worked.
http://kb.imailserver.com/cgi-bin/imail.cfg/php/enduser/std_adp.php?p_faqid=42&p_created=1218038946
> You'll need the full chain in your cert file as explained in this article:
http://support.ipswitch.com/kb/IM-20010425-DM01.htm
-- PORTS AND FIREWALL--
> You'll want to make sure spammers can't inject mail straight into your mail server, so set up your firewall to only allow SMTP to Postini
> In order to let users connect, you set up a separate port for them to connect to and force authentication on that port. In Imail there's an option to do this on the "advanced" link for SMTP server settings.
> I also tried to use that same port for SSL hoping all traffic would be SSL encrypted on that alternate port but found the following info at the bottom of this link:
http://support.ipswitch.com/kb/IM-20050428-DM01.htm
Note for Outlook and Outlook Express users: If you tell your client to use SSL over SMTP and set the port to anything other than 25 the client will attempt to negotiate the SSL connection before sending the EHLO as if it was communicating over a dedicated SSL port. This will not work on the extra port. Users must issue EHLO and then STARTTLS to use SSL over the alternate SMTP port.
When users tried to enter the port and check "SSL" in outlook didn't work. They had to uncheck it to get this to work. However another mail connection I used from an alternate piece of software didn't work. By moving SSL to a separate port I believe that Outlook will attempt to use TLS if available. What I wonder is - what if something happens and it's not? The problem here is that with Postini you can force TLS between your mail server and Postini and between Postini and other mail servers, but for people connecting from their machine to our mail server - we also want to make sure that portion of the transmission is encrypted. Need to set up some alternate type of monitoring to enforce this potentially.
> So for now users are set up to connect to alternate port with SSL unchecked.
> Set up SSL port for POP/IMAP also and provided that port with "server requires SSL" setting.
TLS is cool because it runs on the same port as SMTP so takes up less ports, easier for users to configure and uses TLS when available otherwise not - but for connections from users to the mail server, wondering how that can be enforced and monitored as noted.
-- DICTIONARY ATTACKS --
Shortly after set up we were plagued by "dictionary attacks" (see my twitter account for more on that http://twitter.com/teriradichel
To prevent or at least slow down dictionary attacks - where someone just hits your web server with every possible valid email they can think of in rapid succession until they find a good one - edit SMTP service settings in the box that says "Dictionary" attacks and limit the number of attacks per session, failures, etc.
-- TRANSFERING MAIL TO MAIL SERVER --
If you're transferring users from another mail server you might also be interested in:
How to transfer your users and mail from your old mail server to new:
http://kb.imailserver.com/cgi-bin/imail.cfg/php/enduser/std_adp.php?p_faqid=716&p_created=1247145176&p_sid=gOAbiCfk&p_accessibility=0&p_redirect=&p_lva=&p_sp=cF9zcmNoPTEmcF9zb3J0X2J5PWRmbHQ6MSZwX2dyaWRzb3J0PSZwX3Jvd19jbnQ9MTU4LDE1OCZwX3Byb2RzPTAmcF9jYXRzPTAmcF9wdj0mcF9jdj0mcF9zZWFyY2hfdHlwZT1hbnN3ZXJzLnNlYXJjaF9ubCZwX3BhZ2U9MiZwX3NlYXJjaF90ZXh0PW1vdmUgbWFpbA**&p_li=&p_topview=1
Basically to move mail you transfer all your users over as noted above. Then you can connect via IMAP in outlook to both mail servers, and copy all the mail from the inbox of one server to the inbox of the other. There may be other folders to copy as well depending on what the user has set up.
Probably missing a few things here but that's the gist of it.
Postini
You probably all run some kind of mail software with spam blocking capabilities. However the painful thing is that spammers are out there inserting mail directly into your mail server and overloading spam boxes all the time. This takes up resources on your mail server(s) and clients.
What's kind of neat about Postini is that it's a filtering service in front of your mail server. You can force all inbound and outbound mail to go through Postini if you have your own mail server or a service that offers Postini. By setting up Postini correctly a lot of the garbage bombarding your server can be stopped before it ever gets that far.
Postini was bought out by Google and as you can imagine has a tad bit more mail to analyze than the average company hosting an email system. By passing all the garbage through them first, I would hope they can do some pretty good analytics on mail going through their system and figure out how to weed out the junk, the same way they figure out how to put the best web sites at the top of the search engine results [hopefully!]
Postini blocks viruses, spam and has a bunch of cool features like allowing you to block IP ranges from sending mail to your server. Since a lot of spam comes from certain parts of the world, if you don't do business in those areas you can simply block out entire spam friendly networks completely. I found that blocking out everything but ARIN significantly reduced my spam on a particularly old domain that is probably in every spammer's list in existence.
The other cool thing about Postini is that it allows you to force TLS between two parties - for regulatory copmliance or simply because you want to know your email transmission is encrypted end to end (which can be hard to do).
These are just a few of the highlights of Postini service.
The biggest problem with Postini is that, if you're a small business, getting support is challenging. You don't always get answers and when you do they are vague or not actually answering the question you asked or simply don't resolve the problem. Lately, however, I haven't had a ton of questions which is good. The service has been pretty reliable.
IMail
If you're not interested in spending mega-bucks on Exchange, IMail is a simpler, less expensive option and the company provides very good support. When it comes to email that's important to me because I find email to be a royal pain. As noted in a recent episode of American Greed, spam is the backbone of organized Internet crime. They are all trying to get into your in box to get you to click a link - or something.
When people's email isn't working - they want it fixed. Now. Although I'm pretty handy with Java, open source typically comes with no or minimal support. Additionally open source has so many people contributing that are not quite held accountable in the same way I would expect a company with paid staff to be. Mostly, I'd rather pay a bit more and get someone who can help me fix things fast than futz around with user groups and mailing lists or reverse engineering code to find a fix to a type of technology I find completely annoying because it is hard to secure, unreliable and always under attack. I prefer to deal with email as little as possible. I just want to call someone and get an answer. There is one person at Ipswitch in particular I have talked to over the years who is extremely knowledgeable about IMail and was able to quickly answer all my questions during set up.
For me running on IIS was kind of a pain because I prefer Java and it's another thing to manage, but I've used Imail in the past and it's reliable with good support as mentioned. For those who like and know IIS this would be a plus. Setting up the SSL cert was much easier with IIS than Java. The web site runs a little slow but going to look into how that can be tweaked. Using IMail is, for the most part, a piece of cake. The only thing I found challenging (having a basic understanding of mail servers) was getting the SSL cert installed and finding all the parts and pieces to make it work with Postini.
Set Up
--IMAIL--
> Purchase or get trial at IPSwitch.com
> Install - call for help as needed.
> Note that MX records will be pointed at Postini (see below) not your mail server.
--POSTINI--
> Set up an account and login to the System Administrator
> Click on the Orgs & Users tab
> Select your top level account from the drop down list of orgs
> Click the link (name) of your account (top node in the list)
> Enter a new org name in the box at the top and click the "add" button
> Click on orgs & users tab to get the full list of orgs
> Click on the name of the org you just created to view the settings.
> Click on the "general settings" link 3/4 down the page
> Change "email config type org" to Yes and save [allows mail server settings for this org]
> Click on "DNS Instructions" for MX records to use...
--EDIT YOUR DNS--
> edit your MX records to include the specified servers (note that each domain should have the domainname itself in the MX record - replace the part that says yourdomain with the domain for which you are setting up email)
--POSTINI--
> Click on "domains" under the tabs and add a domain
> Click on "users" under the tabs and createa user
> Click on "Inbound Servers" tab, "Delivery Manager", then "Edit" to add your mail servers. If you have your own mail server choose "Use my own mail server" and % to balance load to each server.
> There are other cool features you can look into later...manually block mail from certain IPs (bottom right when you click on "Inbound Servers"), force TLS between certain domains, etc.
--IMAIL--
> In your mail server allow relay from Postini (ONLY so you don't create an open relay for spammers!) In the latest version of Imail you would click on "Services" on the left, then double click on "Imail SMTP service".
> Under "General" click on the "Addresses" tab. Select "Group" from the drop down and enter the Postini IP addresses specified for your outbound mail configuration. The IP addresses depend on which server number you were assigned when you signed up for the service. The server number is in your MX records specified in the DNS instructions. More in this document:
Postini Outbound Services
--POSTINI--
> Click on "Outbound Servers" tab, and at the bottom left choose "Add Another Outbound Email Server" and add your mail server(s) that will be used for sending bounces back to your users.
--IMAIL --
> Set up webmail per the IMail instructions.
> If you want webmail to show up on it's own domain with no directory at the end and no access to the administrator and other things from that domain, set up a second web site on IIS and point it specifically at the webmail folder instead of the parent directory. In other words instead of mail.mydomain.com/web_mail_folder to get to mail you point IIS web site at the web_mail_folder and then can get to that same thing by going to mail.mydomain.com (if that doesn't make sense...call IPSwitch. They are very helpful.)
-- SSL CERTIFICATE (webmail and mail traffic) --
> In IIS create a cert request. I used Digicert.com and they were extremely easy to work with. Click on the IIS link on the left side of their home page and scroll to the bottom to get instructions for IIS cert request.
> When you get back the certificate, install it in IIS (for the domain you're running webmail on).
> Test your webmail domain and make sure it's accessible via SSL (https://mail.mydomain.com)
> For Digicert there were some chained certificates I needed to install in IMail. Originally exporting from IIS didn't work because the output didn't include the full chain. So...
> Use the MMC to export the cert with the full chain of certs:
http://www.digicert.com/ssl-support/pfx-import-export-iis-7.htm
> These "unsupported" instructions will get you two files that have the private key and the cert in separate files - I don't know if there's any issue with this...but I did it and it worked.
http://kb.imailserver.com/cgi-bin/imail.cfg/php/enduser/std_adp.php?p_faqid=42&p_created=1218038946
> You'll need the full chain in your cert file as explained in this article:
http://support.ipswitch.com/kb/IM-20010425-DM01.htm
-- PORTS AND FIREWALL--
> You'll want to make sure spammers can't inject mail straight into your mail server, so set up your firewall to only allow SMTP to Postini
> In order to let users connect, you set up a separate port for them to connect to and force authentication on that port. In Imail there's an option to do this on the "advanced" link for SMTP server settings.
> I also tried to use that same port for SSL hoping all traffic would be SSL encrypted on that alternate port but found the following info at the bottom of this link:
http://support.ipswitch.com/kb/IM-20050428-DM01.htm
Note for Outlook and Outlook Express users: If you tell your client to use SSL over SMTP and set the port to anything other than 25 the client will attempt to negotiate the SSL connection before sending the EHLO as if it was communicating over a dedicated SSL port. This will not work on the extra port. Users must issue EHLO and then STARTTLS to use SSL over the alternate SMTP port.
When users tried to enter the port and check "SSL" in outlook didn't work. They had to uncheck it to get this to work. However another mail connection I used from an alternate piece of software didn't work. By moving SSL to a separate port I believe that Outlook will attempt to use TLS if available. What I wonder is - what if something happens and it's not? The problem here is that with Postini you can force TLS between your mail server and Postini and between Postini and other mail servers, but for people connecting from their machine to our mail server - we also want to make sure that portion of the transmission is encrypted. Need to set up some alternate type of monitoring to enforce this potentially.
> So for now users are set up to connect to alternate port with SSL unchecked.
> Set up SSL port for POP/IMAP also and provided that port with "server requires SSL" setting.
TLS is cool because it runs on the same port as SMTP so takes up less ports, easier for users to configure and uses TLS when available otherwise not - but for connections from users to the mail server, wondering how that can be enforced and monitored as noted.
-- DICTIONARY ATTACKS --
Shortly after set up we were plagued by "dictionary attacks" (see my twitter account for more on that http://twitter.com/teriradichel
To prevent or at least slow down dictionary attacks - where someone just hits your web server with every possible valid email they can think of in rapid succession until they find a good one - edit SMTP service settings in the box that says "Dictionary" attacks and limit the number of attacks per session, failures, etc.
-- TRANSFERING MAIL TO MAIL SERVER --
If you're transferring users from another mail server you might also be interested in:
How to transfer your users and mail from your old mail server to new:
http://kb.imailserver.com/cgi-bin/imail.cfg/php/enduser/std_adp.php?p_faqid=716&p_created=1247145176&p_sid=gOAbiCfk&p_accessibility=0&p_redirect=&p_lva=&p_sp=cF9zcmNoPTEmcF9zb3J0X2J5PWRmbHQ6MSZwX2dyaWRzb3J0PSZwX3Jvd19jbnQ9MTU4LDE1OCZwX3Byb2RzPTAmcF9jYXRzPTAmcF9wdj0mcF9jdj0mcF9zZWFyY2hfdHlwZT1hbnN3ZXJzLnNlYXJjaF9ubCZwX3BhZ2U9MiZwX3NlYXJjaF90ZXh0PW1vdmUgbWFpbA**&p_li=&p_topview=1
Basically to move mail you transfer all your users over as noted above. Then you can connect via IMAP in outlook to both mail servers, and copy all the mail from the inbox of one server to the inbox of the other. There may be other folders to copy as well depending on what the user has set up.
Probably missing a few things here but that's the gist of it.
Monday, November 01, 2010
JDK version - Eclipse, Maven
Do you know what compiler Maven is really compiling with when you choose "install"?
I am able to get a project in Maven-Eclipse-Plugin to compile with a specified compiler when using "Maven build..." option. But when I choose "Maven install" it has a mind of its own and decides to use an old compiler version. This may cause a project to fail to build due to incompatibilities (generics) or introduce security issues because various security flaws have been fixed in newer versions of Java.
What really bugs me is that this happened without me changing anything as noted in this previous post, and is kind of frustrating because have to spend time futzing with this instead of getting work done:
Maven-Eclipse compiler issue
Didn't get so lucky with some simple tweaking in my last post, so added compiler variable to a profile in settings.xml like this:
Specify Maven Compiler JDK version in settings
I added a new profile to my Maven settings.xml file as specified in link above. I called my compiler variable MY_JAVA_HOME and pointed it to the JDK I want to use.
Then I added the maven compiler plugin to my project by right clicking on the project node in Eclipse, choosing Maven (I'm using the Maven-eclipse-plugin as specified in other posts which is the source of the Maven right click menu in Eclipse), then adding this plugin: maven-compiler-plugin.
I then replaced configuration node as specified above but using the variable I created in my settings.xml profile:
Seems to work, however there is a warning in the above link that this does not necessarily force the compiler to use the specified JDK version (which I find kind of odd also - use my configuration and throw an error if it won't work).
I actually dropped out of Eclipse and ran mvn install from the command line to get a bit more control over what was going on and was able to get all my projects to compile and install again.
I personally don't like when my software applications do things I don't ask them to do and don't tell me about it - and it's not the way I have it configured.
I am able to get a project in Maven-Eclipse-Plugin to compile with a specified compiler when using "Maven build..." option. But when I choose "Maven install" it has a mind of its own and decides to use an old compiler version. This may cause a project to fail to build due to incompatibilities (generics) or introduce security issues because various security flaws have been fixed in newer versions of Java.
What really bugs me is that this happened without me changing anything as noted in this previous post, and is kind of frustrating because have to spend time futzing with this instead of getting work done:
Maven-Eclipse compiler issue
Didn't get so lucky with some simple tweaking in my last post, so added compiler variable to a profile in settings.xml like this:
Specify Maven Compiler JDK version in settings
I added a new profile to my Maven settings.xml file as specified in link above. I called my compiler variable MY_JAVA_HOME and pointed it to the JDK I want to use.
Then I added the maven compiler plugin to my project by right clicking on the project node in Eclipse, choosing Maven (I'm using the Maven-eclipse-plugin as specified in other posts which is the source of the Maven right click menu in Eclipse), then adding this plugin: maven-compiler-plugin.
I then replaced configuration node as specified above but using the variable I created in my settings.xml profile:
<configuration>
<verbose>true</verbose>
<fork>true</fork>
<executable>${MY_JAVA_HOME}/bin/javac</executable>
<compilerVersion>1.6</compilerVersion>
</configuration>
Seems to work, however there is a warning in the above link that this does not necessarily force the compiler to use the specified JDK version (which I find kind of odd also - use my configuration and throw an error if it won't work).
I actually dropped out of Eclipse and ran mvn install from the command line to get a bit more control over what was going on and was able to get all my projects to compile and install again.
I personally don't like when my software applications do things I don't ask them to do and don't tell me about it - and it's not the way I have it configured.
Tuesday, October 05, 2010
Securing Encryption Keys
I see a lot of articles about encryption but not a lot of information about protecting encryption keys. What good does encryption do if someone has the key? That's like locking your house when you leave for work but leaving the key hanging on the door.
Let's say you encrypt a configuration file. Now you start up your app and you need to decrypt that file. In order to decrypt there's a key somewhere most likely that allows decryption of that file. Where is that key stored? Do you have to type it in when you run the application? Do you have to look it up in a database? How do you secure access to that database? Do you put it in another file? How do you secure access to that file over and above the way you secured access to the configuration file itself?
Whoever has access to that key can decrypt your data - do network admins have access to the file with the key? Do database administrators have access to the key stored in the database? Can a hacker access the key through a backdoor or SQL injection under the permisissions of the user account that runs your web application? Is the key sent unprotected over the network or with weak SSL encryption? Is the key floating in memory on your machine in easy to spot format? Is it embedded in a Java class that is simple to decompile using tools on the web? Are your files and databases backed up to a back up device accessible by third parties? How do you implement checks and balances required for PCI Compliance in terms of encryption key storage and retrieval?
Here is some information I was able to find on the topic:
Some suggestions for protecting encryption keys
http://it.toolbox.com/wiki/index.php/Protect_encryption_keys_in_memory,_files,_databases,_and_during_transport_between_system_processes
PCI Encryption suggestions:
http://forums.sun.com/thread.jspa?threadID=5433515
Limited information but discussing the issues around keys:
http://www.symantec.com/connect/articles/introduction-encryption
Encryption key policy from SANS:
http://www.sans.org/security-resources/policies/User_Encryption_Key_Protection_Policy.pdf
Intel loses DRM Protection Key 9/17/2010 - key management is not simple...
http://www.voltage.com/products/data_protection.htm
Voltage has been in the encryption / key management space for a while - but how are your keys secure when given to a third party?
http://www.voltage.com/products/data_protection.htm
Let's say you encrypt a configuration file. Now you start up your app and you need to decrypt that file. In order to decrypt there's a key somewhere most likely that allows decryption of that file. Where is that key stored? Do you have to type it in when you run the application? Do you have to look it up in a database? How do you secure access to that database? Do you put it in another file? How do you secure access to that file over and above the way you secured access to the configuration file itself?
Whoever has access to that key can decrypt your data - do network admins have access to the file with the key? Do database administrators have access to the key stored in the database? Can a hacker access the key through a backdoor or SQL injection under the permisissions of the user account that runs your web application? Is the key sent unprotected over the network or with weak SSL encryption? Is the key floating in memory on your machine in easy to spot format? Is it embedded in a Java class that is simple to decompile using tools on the web? Are your files and databases backed up to a back up device accessible by third parties? How do you implement checks and balances required for PCI Compliance in terms of encryption key storage and retrieval?
Here is some information I was able to find on the topic:
Some suggestions for protecting encryption keys
http://it.toolbox.com/wiki/index.php/Protect_encryption_keys_in_memory,_files,_databases,_and_during_transport_between_system_processes
PCI Encryption suggestions:
http://forums.sun.com/thread.jspa?threadID=5433515
Limited information but discussing the issues around keys:
http://www.symantec.com/connect/articles/introduction-encryption
Encryption key policy from SANS:
http://www.sans.org/security-resources/policies/User_Encryption_Key_Protection_Policy.pdf
Intel loses DRM Protection Key 9/17/2010 - key management is not simple...
http://www.voltage.com/products/data_protection.htm
Voltage has been in the encryption / key management space for a while - but how are your keys secure when given to a third party?
http://www.voltage.com/products/data_protection.htm
Saturday, July 17, 2010
Sunday, July 11, 2010
Maven Mysteries - Generics and Compiler Issues
Maven by default compiles code with a lower compiler version than 1.5 which has support for generics.
Today I was programming along when suddenly a project that has been compiling for a while suddenly stopped compiling (using Maven + Eclipse) and Maven started griping about the project code's inclusion of generics when this is not supported prior to JDK 1.5. Hmm. I had this working before so why did it suddenly start failing? Who knows.
So I go in and reset the project to use the 1.6 compiler by right clicking on the project node in Eclipse, choosing compiler and changing the settings.
No dice.
I then right clicked on the project node, chose Maven (I'm using Maven plugin as described in another post) and choose "Update Project Configuration". Right click on the pom, choose Run As and Maven Install...well then I started getting compiler errors in Eclipse I wasn't getting before. I went to one of the errors, right clicked and told it to change the project to use JDK 1.5 which supports generics.
Ok now it compiles...but now the setting changes I just made say 1.5 instead of 1.6. Why didn't it work when I set the project to use JDK 1.6? So I go back to the project node, change the compiler setting to JDK 1.6. Still compiles. That's good.
Then I go right click and choose Maven and Update Project Configuration. Running Maven Install still works.
OK great. Whatever.
Today I was programming along when suddenly a project that has been compiling for a while suddenly stopped compiling (using Maven + Eclipse) and Maven started griping about the project code's inclusion of generics when this is not supported prior to JDK 1.5. Hmm. I had this working before so why did it suddenly start failing? Who knows.
So I go in and reset the project to use the 1.6 compiler by right clicking on the project node in Eclipse, choosing compiler and changing the settings.
No dice.
I then right clicked on the project node, chose Maven (I'm using Maven plugin as described in another post) and choose "Update Project Configuration". Right click on the pom, choose Run As and Maven Install...well then I started getting compiler errors in Eclipse I wasn't getting before. I went to one of the errors, right clicked and told it to change the project to use JDK 1.5 which supports generics.
Ok now it compiles...but now the setting changes I just made say 1.5 instead of 1.6. Why didn't it work when I set the project to use JDK 1.6? So I go back to the project node, change the compiler setting to JDK 1.6. Still compiles. That's good.
Then I go right click and choose Maven and Update Project Configuration. Running Maven Install still works.
OK great. Whatever.
Classloaders In Depth
Digging into the details of classloads has been on my list of research items for a while. I understand the concepts and the implications - what if a class gets loaded other than the one you intend to be running in your app and you're not really running the code you think you are?
Anyway...ran into this error today...
java.lang.LinkageError: loader constraint violation: when resolving method "[class]" the class loader (instance of [classloader1]) of the current class, [class], and the class loader (instance of [classloader2]) for resolved class, [class], have different Class objects for the type [class in conflict] used in the signature
Which led me to this great blog post which does a great job of epxlaining the nitty gritty of class loaders and potential issues with self-first and parent-first class loading...
Java Class Loaders
Nice.
Oh and by the way, I fixed this by changing the package of the class in my web app so it was different than the package of the conflicting class loaded by the web server. This may not be the right solution in all cases - you may want to remove the class from your web app and use the parent class or vice versa instead.
Anyway...ran into this error today...
java.lang.LinkageError: loader constraint violation: when resolving method "[class]" the class loader (instance of [classloader1]) of the current class, [class], and the class loader (instance of [classloader2]) for resolved class, [class], have different Class objects for the type [class in conflict] used in the signature
Which led me to this great blog post which does a great job of epxlaining the nitty gritty of class loaders and potential issues with self-first and parent-first class loading...
Java Class Loaders
Nice.
Oh and by the way, I fixed this by changing the package of the class in my web app so it was different than the package of the conflicting class loaded by the web server. This may not be the right solution in all cases - you may want to remove the class from your web app and use the parent class or vice versa instead.
Saturday, July 10, 2010
Strange Network / Web Site behavior
Something very odd happened when I transferred web sites from one network to another, and view them using one particular network provider.
If I hit the web sites in rapid succession the images go all wacky and some images will get displayed multiple times when they are only supposed to be displayed once, for instance. These are straight html pages - about as basic as you can get.
I cannot reproduce this skewing of the web pages when working locally on the machine that is hosting the web site.
I also noticed that when I strolled down to the local coffee shop with my laptop I could not reproduce this funky behavior. This leads me to believe that the problem is due to the network or software thereon of the ISP at the particular location having the problem. It seems like something is capturing and altering the results of the request in transit and at some point before reaching my computer they are getting skewed.
Will have to research further.
7/15/2010
Interesting...last night I uncovered that my mail seemed to be going through a proxy server. They claimed that my network traffic was not going out through any proxy servers previously, but I asked them to double check the web traffic since obviously something was affecting the email traffic and sending it where it shouldn't be going.
Magically today, I have not yet been able to reproduce whatever was causing web sites to be skewed. I have not yet tested on Comcast network again.
If I hit the web sites in rapid succession the images go all wacky and some images will get displayed multiple times when they are only supposed to be displayed once, for instance. These are straight html pages - about as basic as you can get.
I cannot reproduce this skewing of the web pages when working locally on the machine that is hosting the web site.
I also noticed that when I strolled down to the local coffee shop with my laptop I could not reproduce this funky behavior. This leads me to believe that the problem is due to the network or software thereon of the ISP at the particular location having the problem. It seems like something is capturing and altering the results of the request in transit and at some point before reaching my computer they are getting skewed.
Will have to research further.
7/15/2010
Interesting...last night I uncovered that my mail seemed to be going through a proxy server. They claimed that my network traffic was not going out through any proxy servers previously, but I asked them to double check the web traffic since obviously something was affecting the email traffic and sending it where it shouldn't be going.
Magically today, I have not yet been able to reproduce whatever was causing web sites to be skewed. I have not yet tested on Comcast network again.
Tuesday, June 29, 2010
Database restore script - roles, users, permissions and schemas
I noted some code for creating some database roles and assigning users to them the other day. Here's a more extensive script that can be used when transfering databases between environments (say dev, test and production) where you want to assign users to specific schemas and roles, clear out permissions, and/or restore database users after restoring a database. Using a script is preferable to doing everything by hand because it can produce more consistent results and will be faster. Additionally you can use the same script in different environments and set up different users and/or schemas as needed.
This script should save a lot of time, though could be even more automated (creating logins for instance).
Additionally if you have existing users with appropriate permissions you may want to (I have recently learned) assign the new user in the newly restored database the old sysid to keep those permissions in tact.
Another item of note - all permissions and roles are stored in the master database. If you want to return to default roles and permissions, hopefully you have backed up the database and can restore the master database to retrieve those settings. Restoring the master database requires putting the SQL Server database in single user mode and there are a few tricks to that which I have around here somewhere...
/*
Before running this script:
a. create the user + pwd on the server
b. replace this everywhere found below with your new user name and schema:
set @newuser = 'test'
set @dbschema = 'dbs'
c. run script below which does the following:
add schema to all dbs
add user to all dbs
add all objects to schema @dbschema
add roles to dbs
add user to roles
*/
/*
STEP 1: add schema to every db
*/
Declare @dbschema varchar(10)
Declare @sql varchar(4000)
set @dbschema = 'dbs'
set @sql = 'IF ''?'' NOT IN (''master'', ''model'', ''msdb'', ''tempdb'') ' +
' BEGIN ' +
' use ? ' +
' declare @s varchar(255) ' +
' if not exists(select name from sys.schemas where name=''' + @dbschema + ''')' +
' begin ' +
' set @s = ''create schema ' + @dbschema +
' '' exec (@s) ' +
' end ' +
' end '
print @sql
exec master.dbo.sp_MSforeachdb @sql
GO
/************
Step 2: change schema of all user defined objects
************/
Declare @sql varchar(2000)
Declare @cntitems int
Declare @dbschema varchar(20)
Declare @oldschema varchar(20)
Declare @itemname varchar(128)
Declare @db varchar(128)
set @dbschema = 'dbs'
if (select object_id('tempdb..##objnames')) is not null
drop table ##objnames
create table ##objnames (itemname varchar(128), oldschema varchar(128), dbname varchar(128))
set @sql = 'IF ''?'' NOT IN (''master'', ''model'', ''msdb'', ''tempdb'') ' +
' BEGIN ' +
' use ? ' +
' insert into ##objnames select t.name, s.name, ''?'' from ' +
' sys.all_objects t ' +
' left outer join sys.schemas s ' +
' on s.schema_id = t.schema_id ' +
' where is_ms_shipped = 0 and parent_object_id = 0 ' +
' and type_desc <> ''system_table'' and ' +
' t.name not like ''dt_%'' and t.name not like ''%#'' and s.name <> ''' + @dbschema +
' '' END'
print @sql
exec master.dbo.sp_MSforeachdb @sql
/*****
loop through the objects we found and change schemas
******/
select @cntitems=count(*) from ##objnames
While @cntitems > 0
begin
select top 1 @itemname=itemname, @oldschema = oldschema, @db = dbname from ##objnames
delete from ##objnames where itemname = @itemname and oldschema=@oldschema and dbname=@db
if (@itemname like '%-%')
print ('ERROR: cannot update item with - in name: ' + @itemname)
else
begin
set @sql = 'use ' + @db + ' ALTER SCHEMA ' + @dbschema + ' TRANSFER ' + @oldschema + '.' + @itemname
print (@sql)
exec (@sql)
end
select @cntitems=count(*) from ##objnames
end
/*********
step 3: delete orphan users now that schemas are fixed
**********/
-- var for exec sql
-- vars are for db loop below
declare @CNT int
declare @name char(128)
declare @sid varbinary(85)
declare @c int
declare @hexnum char(100)
set nocount on
/*
You may need to change the database owner...
set @sql = 'select @cnt = count(*) from master..syslogins l right join ' +
rtrim(@db) + '..sysusers u on l.sid = u.sid' +
' where l.sid is null and u.name = ''DBO'''
exec sp_executesql @sql,N'@cnt int out',@cnt out
if DB is not mapped to login that exists map DBO to SA
if @cnt = 1
begin
--print 'exec ' + @db + '..sp_changedbowner ''SA'''
-- exec sp_changedbowner 'SA'
end -- if @cnt = 1
*/
Declare @sql varchar(1000)
Declare @db varchar(50)
if (select object_id('tempdb..##orphans')) is not null
drop table ##orphans
create table ##orphans (orphan varchar(128), dbname varchar(128))
set @sql= 'BEGIN ' +
' use ? ' +
' insert into ##orphans select u.name, ''?'' from master..syslogins l right join ' +
' sysusers u on l.sid = u.sid ' +
' where l.sid is null and issqlrole <> 1 and isapprole <> 1 ' +
' and (u.name <> ''INFORMATION_SCHEMA'' and u.name <> ''guest'' and u.name <> ''dbo'' ' +
' and u.name <> ''system_function_schema'' ' +
' and u.name <> ''sys'') and u.name <> ''MS_DataCollectorInternalUser'' ' +
' END '
exec master.dbo.sp_MSforeachdb @sql
select @cnt = count(*) from ##orphans
WHILE @cnt > 0
BEGIN
select top 1 @name= orphan, @db = dbname from ##orphans
delete from ##orphans where orphan = @name
set @sql = 'exec ' + rtrim(@db) + '..sp_revokedbaccess ''' + rtrim(@name) + ''''
print @sql
exec (@sql) -- drop the orphan from the db
select @cnt = count(*) from ##orphans
end
go
/**********
Step 4: create roles and assign user to roles
*********/
Declare @newuser varchar(20)
declare @dbschema varchar(20)
Declare @sql varchar(1000)
set @dbschema = 'dbs'
set @newuser = 'test'
/* create the cto roles on each database */
set @sql = 'IF ''?'' NOT IN (''master'', ''model'', ''msdb'', ''tempdb'') ' +
' BEGIN ' +
' use ? ' +
' IF NOT EXISTS (select * from sys.database_principals where name=''dbs_select'') ' +
' create role dbs_select ' +
' end '
print @sql
exec master.dbo.sp_MSforeachdb @sql
set @sql = 'IF ''?'' NOT IN (''master'', ''model'', ''msdb'', ''tempdb'') BEGIN ' +
+ ' use ? ' +
' IF NOT EXISTS (select * from sys.database_principals where name=''dbs_execute'') ' +
'create role dbs_execute ' +
' end '
print @sql
exec master.dbo.sp_MSforeachdb @sql
set @sql = 'IF ''?'' NOT IN (''master'', ''model'', ''msdb'', ''tempdb'') BEGIN ' +
+ ' use ? ' +
' IF NOT EXISTS (select * from sys.database_principals where name=''dbs_delete'') ' +
'create role dbs_delete ' +
' end '
print @sql
exec master.dbo.sp_MSforeachdb @sql
set @sql = 'IF ''?'' NOT IN (''master'', ''model'', ''msdb'', ''tempdb'') ' +
' BEGIN ' +
' use ? ' +
' IF NOT EXISTS (select * from sys.database_principals where name=''dbs_update'') ' +
' create role dbs_update ' +
' end '
print @sql
exec master.dbo.sp_MSforeachdb @sql
set @sql = 'IF ''?'' NOT IN (''master'', ''model'', ''msdb'', ''tempdb'') BEGIN ' +
+ ' use ? ' +
' IF NOT EXISTS (select * from sys.database_principals where name=''dbs_insert'') ' +
' create role dbs_insert ' +
' end '
print @sql
exec master.dbo.sp_MSforeachdb @sql
/* grant these roles permissions to cto schema in this database */
set @sql = 'IF ''?'' NOT IN (''master'', ''model'', ''msdb'', ''tempdb'') ' +
' begin ' +
' use ? ' +
' grant execute on schema :: ' + @dbschema + ' TO dbs_execute ' +
' end '
print (@sql)
exec master.dbo.sp_MSforeachdb @sql
set @sql = 'IF ''?'' NOT IN (''master'', ''model'', ''msdb'', ''tempdb'') ' +
' begin ' +
' use ? ' +
' print ''?'' '+
' grant select on schema :: ' + @dbschema + ' TO dbs_select ' +
' end '
print (@sql)
exec master.dbo.sp_MSforeachdb @sql
set @sql = 'IF ''?'' NOT IN (''master'', ''model'', ''msdb'', ''tempdb'') ' +
' begin ' +
' use ? ' +
' grant delete on schema :: ' + @dbschema + ' TO dbs_delete ' +
' end '
print (@sql)
exec master.dbo.sp_MSforeachdb @sql
set @sql = 'IF ''?'' NOT IN (''master'', ''model'', ''msdb'', ''tempdb'') ' +
' begin ' +
' use ? ' +
' grant insert on schema :: ' + @dbschema + ' TO dbs_insert ' +
' end '
print (@sql)
exec master.dbo.sp_MSforeachdb @sql
set @sql = 'IF ''?'' NOT IN (''master'', ''model'', ''msdb'', ''tempdb'') ' +
' begin ' +
' use ? ' +
' grant update on schema :: ' + @dbschema + ' TO dbs_update ' +
' end '
print (@sql)
exec master.dbo.sp_MSforeachdb @sql
set @sql =
'IF ''?'' NOT IN (''master'', ''model'', ''msdb'', ''tempdb'') ' +
' begin ' +
' use ? ' +
'if not exists( select u.name from master..syslogins l right join ' +
+ '?.dbo.sysusers u
on l.sid = u.sid
where u.name=''' + @newuser + ''') create user ' + @newuser
+ ' end '
print (@sql)
exec master.dbo.sp_MSforeachdb @sql
/* assign user to roles in each db */
set @sql = 'IF ''?'' NOT IN (''master'', ''model'', ''msdb'', ''tempdb'') BEGIN ' +
' use ? ' +
' exec sp_addrolemember ''dbs_execute'', ' + @newuser +
' end '
print @sql
exec master.dbo.sp_MSforeachdb @sql
set @sql = 'IF ''?'' NOT IN (''master'', ''model'', ''msdb'', ''tempdb'') BEGIN ' +
' use ? ' +
' exec sp_addrolemember ''dbs_select'', ' + @newuser +
' end '
print (@sql)
exec master.dbo.sp_MSforeachdb @sql
set @sql = 'IF ''?'' NOT IN (''master'', ''model'', ''msdb'', ''tempdb'') BEGIN ' +
' use ? ' +
' exec sp_addrolemember ''dbs_insert'', ' + @newuser +
' end '
print (@sql)
exec master.dbo.sp_MSforeachdb @sql
set @sql = 'IF ''?'' NOT IN (''master'', ''model'', ''msdb'', ''tempdb'') BEGIN ' +
' use ? ' +
' exec sp_addrolemember ''dbs_update'', ' + @newuser +
' end '
print (@sql)
exec master.dbo.sp_MSforeachdb @sql
set @sql = 'IF ''?'' NOT IN (''master'', ''model'', ''msdb'', ''tempdb'') BEGIN ' +
' use ? ' +
' exec sp_addrolemember ''dbs_delete'', ' + @newuser +
' end '
print (@sql)
exec master.dbo.sp_MSforeachdb @sql
set @newuser='test'
set @dbschema = 'dbs'
set @sql = 'IF ''?'' NOT IN (''master'', ''model'', ''msdb'', ''tempdb'') ' +
' begin ' +
' use ? ' +
' declare @s varchar(255) ' +
' set @s = ''alter user ' + @newuser + ' with default_schema = ' + @dbschema + ' '' ' +
' exec (@s) ' +
' end '
print (@sql)
exec master.dbo.sp_MSforeachdb @sql
-- Remove temporary tables
drop table ##orphans, ##objnames
This script should save a lot of time, though could be even more automated (creating logins for instance).
Additionally if you have existing users with appropriate permissions you may want to (I have recently learned) assign the new user in the newly restored database the old sysid to keep those permissions in tact.
Another item of note - all permissions and roles are stored in the master database. If you want to return to default roles and permissions, hopefully you have backed up the database and can restore the master database to retrieve those settings. Restoring the master database requires putting the SQL Server database in single user mode and there are a few tricks to that which I have around here somewhere...
/*
Before running this script:
a. create the user + pwd on the server
b. replace this everywhere found below with your new user name and schema:
set @newuser = 'test'
set @dbschema = 'dbs'
c. run script below which does the following:
add schema to all dbs
add user to all dbs
add all objects to schema @dbschema
add roles to dbs
add user to roles
*/
/*
STEP 1: add schema to every db
*/
Declare @dbschema varchar(10)
Declare @sql varchar(4000)
set @dbschema = 'dbs'
set @sql = 'IF ''?'' NOT IN (''master'', ''model'', ''msdb'', ''tempdb'') ' +
' BEGIN ' +
' use ? ' +
' declare @s varchar(255) ' +
' if not exists(select name from sys.schemas where name=''' + @dbschema + ''')' +
' begin ' +
' set @s = ''create schema ' + @dbschema +
' '' exec (@s) ' +
' end ' +
' end '
print @sql
exec master.dbo.sp_MSforeachdb @sql
GO
/************
Step 2: change schema of all user defined objects
************/
Declare @sql varchar(2000)
Declare @cntitems int
Declare @dbschema varchar(20)
Declare @oldschema varchar(20)
Declare @itemname varchar(128)
Declare @db varchar(128)
set @dbschema = 'dbs'
if (select object_id('tempdb..##objnames')) is not null
drop table ##objnames
create table ##objnames (itemname varchar(128), oldschema varchar(128), dbname varchar(128))
set @sql = 'IF ''?'' NOT IN (''master'', ''model'', ''msdb'', ''tempdb'') ' +
' BEGIN ' +
' use ? ' +
' insert into ##objnames select t.name, s.name, ''?'' from ' +
' sys.all_objects t ' +
' left outer join sys.schemas s ' +
' on s.schema_id = t.schema_id ' +
' where is_ms_shipped = 0 and parent_object_id = 0 ' +
' and type_desc <> ''system_table'' and ' +
' t.name not like ''dt_%'' and t.name not like ''%#'' and s.name <> ''' + @dbschema +
' '' END'
print @sql
exec master.dbo.sp_MSforeachdb @sql
/*****
loop through the objects we found and change schemas
******/
select @cntitems=count(*) from ##objnames
While @cntitems > 0
begin
select top 1 @itemname=itemname, @oldschema = oldschema, @db = dbname from ##objnames
delete from ##objnames where itemname = @itemname and oldschema=@oldschema and dbname=@db
if (@itemname like '%-%')
print ('ERROR: cannot update item with - in name: ' + @itemname)
else
begin
set @sql = 'use ' + @db + ' ALTER SCHEMA ' + @dbschema + ' TRANSFER ' + @oldschema + '.' + @itemname
print (@sql)
exec (@sql)
end
select @cntitems=count(*) from ##objnames
end
/*********
step 3: delete orphan users now that schemas are fixed
**********/
-- var for exec sql
-- vars are for db loop below
declare @CNT int
declare @name char(128)
declare @sid varbinary(85)
declare @c int
declare @hexnum char(100)
set nocount on
/*
You may need to change the database owner...
set @sql = 'select @cnt = count(*) from master..syslogins l right join ' +
rtrim(@db) + '..sysusers u on l.sid = u.sid' +
' where l.sid is null and u.name = ''DBO'''
exec sp_executesql @sql,N'@cnt int out',@cnt out
if DB is not mapped to login that exists map DBO to SA
if @cnt = 1
begin
--print 'exec ' + @db + '..sp_changedbowner ''SA'''
-- exec sp_changedbowner 'SA'
end -- if @cnt = 1
*/
Declare @sql varchar(1000)
Declare @db varchar(50)
if (select object_id('tempdb..##orphans')) is not null
drop table ##orphans
create table ##orphans (orphan varchar(128), dbname varchar(128))
set @sql= 'BEGIN ' +
' use ? ' +
' insert into ##orphans select u.name, ''?'' from master..syslogins l right join ' +
' sysusers u on l.sid = u.sid ' +
' where l.sid is null and issqlrole <> 1 and isapprole <> 1 ' +
' and (u.name <> ''INFORMATION_SCHEMA'' and u.name <> ''guest'' and u.name <> ''dbo'' ' +
' and u.name <> ''system_function_schema'' ' +
' and u.name <> ''sys'') and u.name <> ''MS_DataCollectorInternalUser'' ' +
' END '
exec master.dbo.sp_MSforeachdb @sql
select @cnt = count(*) from ##orphans
WHILE @cnt > 0
BEGIN
select top 1 @name= orphan, @db = dbname from ##orphans
delete from ##orphans where orphan = @name
set @sql = 'exec ' + rtrim(@db) + '..sp_revokedbaccess ''' + rtrim(@name) + ''''
print @sql
exec (@sql) -- drop the orphan from the db
select @cnt = count(*) from ##orphans
end
go
/**********
Step 4: create roles and assign user to roles
*********/
Declare @newuser varchar(20)
declare @dbschema varchar(20)
Declare @sql varchar(1000)
set @dbschema = 'dbs'
set @newuser = 'test'
/* create the cto roles on each database */
set @sql = 'IF ''?'' NOT IN (''master'', ''model'', ''msdb'', ''tempdb'') ' +
' BEGIN ' +
' use ? ' +
' IF NOT EXISTS (select * from sys.database_principals where name=''dbs_select'') ' +
' create role dbs_select ' +
' end '
print @sql
exec master.dbo.sp_MSforeachdb @sql
set @sql = 'IF ''?'' NOT IN (''master'', ''model'', ''msdb'', ''tempdb'') BEGIN ' +
+ ' use ? ' +
' IF NOT EXISTS (select * from sys.database_principals where name=''dbs_execute'') ' +
'create role dbs_execute ' +
' end '
print @sql
exec master.dbo.sp_MSforeachdb @sql
set @sql = 'IF ''?'' NOT IN (''master'', ''model'', ''msdb'', ''tempdb'') BEGIN ' +
+ ' use ? ' +
' IF NOT EXISTS (select * from sys.database_principals where name=''dbs_delete'') ' +
'create role dbs_delete ' +
' end '
print @sql
exec master.dbo.sp_MSforeachdb @sql
set @sql = 'IF ''?'' NOT IN (''master'', ''model'', ''msdb'', ''tempdb'') ' +
' BEGIN ' +
' use ? ' +
' IF NOT EXISTS (select * from sys.database_principals where name=''dbs_update'') ' +
' create role dbs_update ' +
' end '
print @sql
exec master.dbo.sp_MSforeachdb @sql
set @sql = 'IF ''?'' NOT IN (''master'', ''model'', ''msdb'', ''tempdb'') BEGIN ' +
+ ' use ? ' +
' IF NOT EXISTS (select * from sys.database_principals where name=''dbs_insert'') ' +
' create role dbs_insert ' +
' end '
print @sql
exec master.dbo.sp_MSforeachdb @sql
/* grant these roles permissions to cto schema in this database */
set @sql = 'IF ''?'' NOT IN (''master'', ''model'', ''msdb'', ''tempdb'') ' +
' begin ' +
' use ? ' +
' grant execute on schema :: ' + @dbschema + ' TO dbs_execute ' +
' end '
print (@sql)
exec master.dbo.sp_MSforeachdb @sql
set @sql = 'IF ''?'' NOT IN (''master'', ''model'', ''msdb'', ''tempdb'') ' +
' begin ' +
' use ? ' +
' print ''?'' '+
' grant select on schema :: ' + @dbschema + ' TO dbs_select ' +
' end '
print (@sql)
exec master.dbo.sp_MSforeachdb @sql
set @sql = 'IF ''?'' NOT IN (''master'', ''model'', ''msdb'', ''tempdb'') ' +
' begin ' +
' use ? ' +
' grant delete on schema :: ' + @dbschema + ' TO dbs_delete ' +
' end '
print (@sql)
exec master.dbo.sp_MSforeachdb @sql
set @sql = 'IF ''?'' NOT IN (''master'', ''model'', ''msdb'', ''tempdb'') ' +
' begin ' +
' use ? ' +
' grant insert on schema :: ' + @dbschema + ' TO dbs_insert ' +
' end '
print (@sql)
exec master.dbo.sp_MSforeachdb @sql
set @sql = 'IF ''?'' NOT IN (''master'', ''model'', ''msdb'', ''tempdb'') ' +
' begin ' +
' use ? ' +
' grant update on schema :: ' + @dbschema + ' TO dbs_update ' +
' end '
print (@sql)
exec master.dbo.sp_MSforeachdb @sql
set @sql =
'IF ''?'' NOT IN (''master'', ''model'', ''msdb'', ''tempdb'') ' +
' begin ' +
' use ? ' +
'if not exists( select u.name from master..syslogins l right join ' +
+ '?.dbo.sysusers u
on l.sid = u.sid
where u.name=''' + @newuser + ''') create user ' + @newuser
+ ' end '
print (@sql)
exec master.dbo.sp_MSforeachdb @sql
/* assign user to roles in each db */
set @sql = 'IF ''?'' NOT IN (''master'', ''model'', ''msdb'', ''tempdb'') BEGIN ' +
' use ? ' +
' exec sp_addrolemember ''dbs_execute'', ' + @newuser +
' end '
print @sql
exec master.dbo.sp_MSforeachdb @sql
set @sql = 'IF ''?'' NOT IN (''master'', ''model'', ''msdb'', ''tempdb'') BEGIN ' +
' use ? ' +
' exec sp_addrolemember ''dbs_select'', ' + @newuser +
' end '
print (@sql)
exec master.dbo.sp_MSforeachdb @sql
set @sql = 'IF ''?'' NOT IN (''master'', ''model'', ''msdb'', ''tempdb'') BEGIN ' +
' use ? ' +
' exec sp_addrolemember ''dbs_insert'', ' + @newuser +
' end '
print (@sql)
exec master.dbo.sp_MSforeachdb @sql
set @sql = 'IF ''?'' NOT IN (''master'', ''model'', ''msdb'', ''tempdb'') BEGIN ' +
' use ? ' +
' exec sp_addrolemember ''dbs_update'', ' + @newuser +
' end '
print (@sql)
exec master.dbo.sp_MSforeachdb @sql
set @sql = 'IF ''?'' NOT IN (''master'', ''model'', ''msdb'', ''tempdb'') BEGIN ' +
' use ? ' +
' exec sp_addrolemember ''dbs_delete'', ' + @newuser +
' end '
print (@sql)
exec master.dbo.sp_MSforeachdb @sql
set @newuser='test'
set @dbschema = 'dbs'
set @sql = 'IF ''?'' NOT IN (''master'', ''model'', ''msdb'', ''tempdb'') ' +
' begin ' +
' use ? ' +
' declare @s varchar(255) ' +
' set @s = ''alter user ' + @newuser + ' with default_schema = ' + @dbschema + ' '' ' +
' exec (@s) ' +
' end '
print (@sql)
exec master.dbo.sp_MSforeachdb @sql
-- Remove temporary tables
drop table ##orphans, ##objnames
Saturday, June 12, 2010
The statement did not return a resultset - bug: MS JDBC driver + SQL Server 2000
There is a bug that has been driving me insane. I thought I was doing something wrong but apparently it's a bug in Microsoft SQL Server JDBC driver when using SQL Server 2000.
I had a bunch of sprocs that were failing for no apparent reason. I thought it was code change to incorporate a database connection pool. Coincidentally I updated the Microsoft JDBC driver at the same time and forgot about it.
Oddly enough the fix is to add "SET NOCOUNT ON" to stored procedures and then things run as expected.
The statement did not resturn a ResultSet
I had a bunch of sprocs that were failing for no apparent reason. I thought it was code change to incorporate a database connection pool. Coincidentally I updated the Microsoft JDBC driver at the same time and forgot about it.
Oddly enough the fix is to add "SET NOCOUNT ON" to stored procedures and then things run as expected.
The statement did not resturn a ResultSet
The Resultset has No Current Row
If you get this error using Microsoft JDBC driver to return records as a ResultSet:
The ResultSet has no current row
ResultSet rs = getResultSetFunction();
rs.next() <-----------
"ResultSet has not current row" does not mean that the resultset has not returned any records. It means that the recordset has not been positioned to the row you want to read from. When you first load the recordset, it's not positioned to any row.
So rs.next() will position the resultset to the first row (i.e. make the first row the current row), so you can read some data from that row.
The result set needs to have a current row to read some data from it.
The ResultSet has no current row
ResultSet rs = getResultSetFunction();
rs.next() <-----------
"ResultSet has not current row" does not mean that the resultset has not returned any records. It means that the recordset has not been positioned to the row you want to read from. When you first load the recordset, it's not positioned to any row.
So rs.next() will position the resultset to the first row (i.e. make the first row the current row), so you can read some data from that row.
The result set needs to have a current row to read some data from it.
Saturday, May 29, 2010
Java Map Collection Classes
Choosing the right collection class has an impact on the performance of your application. In a previous post I gave a run down of Big O Notation. A hash map look up will have better performance generally than other types of collections when you can look up the data based on a key - O(1) since the underlying data structure is an array - and is useful when the order of the data is not important.
This article has a good run down of Java Maps including how they work and how to choose the right one for your application:
Java Map Collection Classes
In general the concept of optimizing for performance after the application has been built - by actually profiling the application and then making specific implementation decisions - is something I liked to hear. You need to understand the basics to implement a solid architecture but true optimization happens after you can measure the performance, taking all application variables into account from network to hardware to various software components and data distribution, and determining where your bottlenecks lie once you can actually measure the results of your particular application.
This article has a good run down of Java Maps including how they work and how to choose the right one for your application:
Java Map Collection Classes
In general the concept of optimizing for performance after the application has been built - by actually profiling the application and then making specific implementation decisions - is something I liked to hear. You need to understand the basics to implement a solid architecture but true optimization happens after you can measure the performance, taking all application variables into account from network to hardware to various software components and data distribution, and determining where your bottlenecks lie once you can actually measure the results of your particular application.
Sunday, May 23, 2010
AJAX Attack Surface
I have read articles stating Ajax *supposedly* does not increase the attack surface of a web application.
Whether or not that statement is true, I've personally been cautious about integrating Ajax into web applications I've written. This isn't because I have anything against Ajax or think people shouldn't use it or that I think there's anything wrong with the technology - the concept is great. In fact I implemented an Ajax like client side very complex quote calculation JavaScript library of sorts for a web site prior to ever hearing of Ajax many years ago to reduce round trips to the server and improve performance for web visitors.
The reason for my caution is that I wanted to completely understand how Ajax functions -- and how to secure applications using it -- before integrating it. To prevent lost sales and problems for my customers, it is easier and just as effective to just leave it out for most web sites which don't have a good justification for using it. For web sites that perform very well, get great search engine rankings and drive lots of business to those customers (and isn't that the point?) without it, Ajax may only be a source of new problems and expense that is not worth the cool factor.
However I have had customers who insist on Ajax and there are times where it does make sense - especially for applications with complex user interfaces that might otherwise use frames, for example, to prevent going back to the server for the same data over and over again - so have been revisiting and working with Ajax lately and researching the security issues surrounding it.
I just read an article by a "white hat" security web site claiming Ajax does not increase the attack surface of a web application. It has very high search engine rankings presumably because all the people who don't understand or care about the security implications of all the new "cool" technology they throw on their web sites are linking to it to prove to the world that Ajax doesn't pose any additional security problems because that's what someone wrote - without really understanding or thinking about the underlying architectural changes that result from using Ajax.
Well, it's probably a semantic argument because the point is, your web site needs to be secure and you need to constantly monitor both your web site and the security newsgroups and publications for new threats. And yes, using Ajax is probably fine and any web technology you choose to use needs to be implemented securely.
However I disagree that Ajax does not increase the attack surface on a web application. Here's why:
#1. Definition of attack surface:
Attack Surface
The attack surface of a software environment is the code within a computer system that can be run by unauthenticated users.
In other words, more code, components and entry points = greater attack surface.
#2. How you reduce attack surface (same article):
The basic strategies of attack surface reduction are to reduce the amount of code running, reduce entry points available...
#3. New object created by third party - Ajax requires Ajax request object which you don't need if not using Ajax. That means you've introducing a new component and more code - which increases the attack surface. Every new thing you add to a software system is a potential new security problem no matter where it originates. That is the nature of software and systems - not because any particular thing you add has inherent security flaws (a separate issue and some things are riskier than others).
#4. More code (most likely) to support Ajax server side. Chances are pretty good that incorporating Ajax into your code base introduces server side code to support smaller requests server side that handle updating subsections of pages.
#5. More client side code. When someone clicks on something that initiates an Ajax request they are running client side code to make the request. That code wasn't needed before Ajax.
#6. Another entry point. Code is going through client side requests handled initially by JavaScript now instead of just being a link clicked on in the browser that is parsed into a request by the browser and sent over. In addition to the browser parsing you now have some code written by the web developer that has to decipher the request and do the right thing with it, potentially check additional user input, and handle errors appropriately. This is an additional step that wasn't there without Ajax. Probably you have new entry points server side as well for more, smaller requests to the web server.
Besides all that...
The thing that I am most concerned about is JavaScript code running client side that, if errors out before gets to the web server, is an error you'll never know about unless the programmer captures the error and sends it back to the server. This is something many sites don't do - including some large company web sites - because if they were doing it correctly I would never see JavaScript errors. And I do all the time.
Client side errors (in the most case without complex error logging or if your error logging fails) won't be in your web server logs and is a clear weak point that hackers will look at to attack your web application in ways that can be hidden from your view. They don't have to cover their tracks in logs. The person using the web site and the person running it may never know anything went wrong.
For example, if a hacker was able to exploit your JavaScript client side to redirect customers making purchases on your site to an alternate site instead so they get the sale, if it all happened on the client machine you'll think the customer just dropped out during the purchase. You'll have no tracks on your web server, no fingerprints, no clues. The customer may have gone onto make a purchase and received what they bought so they never knew there was a problem either. The other thing is code can be introduced to execute client side functions behind the scenes that don't go to the web server and don't show the user that anything is going on.
So, I would say Ajax increases the attack surface, but even if that technically is not true due to some semantically technical, philosophical justification - Ajax makes applications more complex with trickier security problems (maintenance issues, and bugs). In my opinion.
That doesn't mean I'm not using it. I'm just trying to figure out how to use it correctly and think too many people throw doodads and cool new technology on their web sites without really understanding the implications of using it and how to implement it in a secure manner.
Whether or not that statement is true, I've personally been cautious about integrating Ajax into web applications I've written. This isn't because I have anything against Ajax or think people shouldn't use it or that I think there's anything wrong with the technology - the concept is great. In fact I implemented an Ajax like client side very complex quote calculation JavaScript library of sorts for a web site prior to ever hearing of Ajax many years ago to reduce round trips to the server and improve performance for web visitors.
The reason for my caution is that I wanted to completely understand how Ajax functions -- and how to secure applications using it -- before integrating it. To prevent lost sales and problems for my customers, it is easier and just as effective to just leave it out for most web sites which don't have a good justification for using it. For web sites that perform very well, get great search engine rankings and drive lots of business to those customers (and isn't that the point?) without it, Ajax may only be a source of new problems and expense that is not worth the cool factor.
However I have had customers who insist on Ajax and there are times where it does make sense - especially for applications with complex user interfaces that might otherwise use frames, for example, to prevent going back to the server for the same data over and over again - so have been revisiting and working with Ajax lately and researching the security issues surrounding it.
I just read an article by a "white hat" security web site claiming Ajax does not increase the attack surface of a web application. It has very high search engine rankings presumably because all the people who don't understand or care about the security implications of all the new "cool" technology they throw on their web sites are linking to it to prove to the world that Ajax doesn't pose any additional security problems because that's what someone wrote - without really understanding or thinking about the underlying architectural changes that result from using Ajax.
Well, it's probably a semantic argument because the point is, your web site needs to be secure and you need to constantly monitor both your web site and the security newsgroups and publications for new threats. And yes, using Ajax is probably fine and any web technology you choose to use needs to be implemented securely.
However I disagree that Ajax does not increase the attack surface on a web application. Here's why:
#1. Definition of attack surface:
Attack Surface
The attack surface of a software environment is the code within a computer system that can be run by unauthenticated users.
In other words, more code, components and entry points = greater attack surface.
#2. How you reduce attack surface (same article):
The basic strategies of attack surface reduction are to reduce the amount of code running, reduce entry points available...
#3. New object created by third party - Ajax requires Ajax request object which you don't need if not using Ajax. That means you've introducing a new component and more code - which increases the attack surface. Every new thing you add to a software system is a potential new security problem no matter where it originates. That is the nature of software and systems - not because any particular thing you add has inherent security flaws (a separate issue and some things are riskier than others).
#4. More code (most likely) to support Ajax server side. Chances are pretty good that incorporating Ajax into your code base introduces server side code to support smaller requests server side that handle updating subsections of pages.
#5. More client side code. When someone clicks on something that initiates an Ajax request they are running client side code to make the request. That code wasn't needed before Ajax.
#6. Another entry point. Code is going through client side requests handled initially by JavaScript now instead of just being a link clicked on in the browser that is parsed into a request by the browser and sent over. In addition to the browser parsing you now have some code written by the web developer that has to decipher the request and do the right thing with it, potentially check additional user input, and handle errors appropriately. This is an additional step that wasn't there without Ajax. Probably you have new entry points server side as well for more, smaller requests to the web server.
Besides all that...
The thing that I am most concerned about is JavaScript code running client side that, if errors out before gets to the web server, is an error you'll never know about unless the programmer captures the error and sends it back to the server. This is something many sites don't do - including some large company web sites - because if they were doing it correctly I would never see JavaScript errors. And I do all the time.
Client side errors (in the most case without complex error logging or if your error logging fails) won't be in your web server logs and is a clear weak point that hackers will look at to attack your web application in ways that can be hidden from your view. They don't have to cover their tracks in logs. The person using the web site and the person running it may never know anything went wrong.
For example, if a hacker was able to exploit your JavaScript client side to redirect customers making purchases on your site to an alternate site instead so they get the sale, if it all happened on the client machine you'll think the customer just dropped out during the purchase. You'll have no tracks on your web server, no fingerprints, no clues. The customer may have gone onto make a purchase and received what they bought so they never knew there was a problem either. The other thing is code can be introduced to execute client side functions behind the scenes that don't go to the web server and don't show the user that anything is going on.
So, I would say Ajax increases the attack surface, but even if that technically is not true due to some semantically technical, philosophical justification - Ajax makes applications more complex with trickier security problems (maintenance issues, and bugs). In my opinion.
That doesn't mean I'm not using it. I'm just trying to figure out how to use it correctly and think too many people throw doodads and cool new technology on their web sites without really understanding the implications of using it and how to implement it in a secure manner.
Files Altered On Web Request
Just experienced strangeness with JavaScript in browser which has various maintenance and security implications.
While testing a web site using Jetty 6.1.12 and Eclipse with Jetty plugin, I noticed that somehow magically after requesting a web page, my files were ending up with extra characters at the end and I wasn't editing them. This happened both with HTML files and JavaScript files. It looked like the file would get the last 150 or so characters at the end of the document copied and pasted to the end of it. I would fix the file. Open it up to verify my fixes were saved. Then I would start and run the application and the problem would happen all over again.
This is odd - The source files shouldn't get edited and saved to disk when I make request to the web site. What would be changing them - the browser? Jetty? Jetty Eclipse Plugin? The OS? Browser plugin? Virus scanning software? Something in network? This is where it gets tricky.
The source files shouldn't get edited and saved to disk when I make request to the local web site. That should never alter the source code in the original web site. The original source files were altered, not just the code shown by view source in the browser or cached JavaScript source.
I figured out a few of things:
1. Could fix the files (delete the extra characters) in WordPad, but NotePad wouldn't make the changes stick for some reasons. I also realized later that NotePad wasn't alerting me when it didn't save the file if locked. WordPad would tell me. I had to shut down the web server to change the files even after setting ofline storage to 0 in FireFox.
2. After fixing the files they would continue to get corrupted. I finally made them read only to try to resolve the problem and changed my logging to indicate any time the files changed or there was an access error to try to pinpoint what was changing them. Even when the files were read only they still got changed and no logs indicating how they were changed or what changed them.
3. JavaScript functions in HTML file and did not have issues and the page loaded consistently on a few tests (did not test extensively).
4. FireFox does not log an error if your JavaScript src file path is incorrect. It just tells you a particular function is missing.
5. I realized even though I thought I turned off Firefox caching was still caching JavaScript files. Closing and opening browser would ensure I was getting the right file.
6. At one point I noticed that I had two opening head tags instead of correctly closing the head tag. That may have caused errors - but shouldn't have caused my source files to get altered.
7. I started working offline to remove any issues related to things getting cached somewhere related to being on the network even though the web page I am requesting is localhost.
8. When I initially had this problem I checked in both IE and FireFox and the same thing seemed to be happening, but perhaps one of them was altering the file so when I checked in the other I got the same result.
I don't know which of the above actually solved the problem but I'm not seeing it anymore and don't have time to explore this further. Just posting it online in case the makers of all this software want to figure out what in the world caused this because having software rewrite your source files when a web page is requested - is a bit troubling.
While testing a web site using Jetty 6.1.12 and Eclipse with Jetty plugin, I noticed that somehow magically after requesting a web page, my files were ending up with extra characters at the end and I wasn't editing them. This happened both with HTML files and JavaScript files. It looked like the file would get the last 150 or so characters at the end of the document copied and pasted to the end of it. I would fix the file. Open it up to verify my fixes were saved. Then I would start and run the application and the problem would happen all over again.
This is odd - The source files shouldn't get edited and saved to disk when I make request to the web site. What would be changing them - the browser? Jetty? Jetty Eclipse Plugin? The OS? Browser plugin? Virus scanning software? Something in network? This is where it gets tricky.
The source files shouldn't get edited and saved to disk when I make request to the local web site. That should never alter the source code in the original web site. The original source files were altered, not just the code shown by view source in the browser or cached JavaScript source.
I figured out a few of things:
1. Could fix the files (delete the extra characters) in WordPad, but NotePad wouldn't make the changes stick for some reasons. I also realized later that NotePad wasn't alerting me when it didn't save the file if locked. WordPad would tell me. I had to shut down the web server to change the files even after setting ofline storage to 0 in FireFox.
2. After fixing the files they would continue to get corrupted. I finally made them read only to try to resolve the problem and changed my logging to indicate any time the files changed or there was an access error to try to pinpoint what was changing them. Even when the files were read only they still got changed and no logs indicating how they were changed or what changed them.
3. JavaScript functions in HTML file and did not have issues and the page loaded consistently on a few tests (did not test extensively).
4. FireFox does not log an error if your JavaScript src file path is incorrect. It just tells you a particular function is missing.
5. I realized even though I thought I turned off Firefox caching was still caching JavaScript files. Closing and opening browser would ensure I was getting the right file.
6. At one point I noticed that I had two opening head tags instead of correctly closing the head tag. That may have caused errors - but shouldn't have caused my source files to get altered.
7. I started working offline to remove any issues related to things getting cached somewhere related to being on the network even though the web page I am requesting is localhost.
8. When I initially had this problem I checked in both IE and FireFox and the same thing seemed to be happening, but perhaps one of them was altering the file so when I checked in the other I got the same result.
I don't know which of the above actually solved the problem but I'm not seeing it anymore and don't have time to explore this further. Just posting it online in case the makers of all this software want to figure out what in the world caused this because having software rewrite your source files when a web page is requested - is a bit troubling.
Tuesday, May 18, 2010
Generic Ajax Request
Here's a generic way to create an AJAX request (AJAX code below being JavaScript):
And a sample function that uses it:
Next on the reading list:
Ajax Security
function createRequest(){
request=null;
try{
request = new XMLHttpRequest();
}catch(tryMS){
try{
request = new ActiveXObject("Mxsml2.XMLHTTP");
}catch(otherMS){
try{
request = new ActiveXObject("Microsoft.XMLHTTP");
}catch(failed){
request=null;
}
}
}
return request;
}
And a sample function that uses it:
function getPage(){
try{
request = createRequest();
var url ="/some/page.html";
request.open("GET", url, true);
request.onreadystatechange=displayPage;
request.send(null);
}catch(e){
alert(e)
}
}
function displayPage(){
try{
if (request.readyState== 4){
obj = document.getElementById('someDiv');
if (request.status== 200){
obj.innerHTML = request.responseText;
}
}
}catch(e){
alert(e + request.responseText);
}
}
Next on the reading list:
Ajax Security
Remove file association with .lnk files in Windows 2008
Why Microsoft allows you to change all .lnk files to be associated with Notepad in Windows 2008 (or any other OS for that matter) I'll never know. Obviously that is going to royally mess things up. It's easy to do also because I just did it - I actually couldn't see I was opening a .lnk file because it had a really long path and in Windows 2008 the interface is funky (don't really like it). I right clicked and chose to open with Notepad. I guess the box was automatically checked to open all files of that sort with Notepad (I didn't check it).
Great. Now all my icons on the whole system show Notepad. When I clicked on any short cut it opened the short cut file in Notepad and displayed gibberish instead of opening the related program. So I thought ok, I'll just go into file associations and restore to default. Nope. Not so simple. There's no option to unassociate the file or restore to default (that I saw at first glance).
Sheesh.
So I just guessed that this might be in the registry and because I live my life on the edge, I searched for the .lnk file association and found it. Actually don't live that dangerously because I compared the settings to another machine that was working and determined that I needed to delete the "OpenWithList" node. Then log out and log back in.
Ouilla.
Great. Now all my icons on the whole system show Notepad. When I clicked on any short cut it opened the short cut file in Notepad and displayed gibberish instead of opening the related program. So I thought ok, I'll just go into file associations and restore to default. Nope. Not so simple. There's no option to unassociate the file or restore to default (that I saw at first glance).
Sheesh.
So I just guessed that this might be in the registry and because I live my life on the edge, I searched for the .lnk file association and found it. Actually don't live that dangerously because I compared the settings to another machine that was working and determined that I needed to delete the "OpenWithList" node. Then log out and log back in.
Ouilla.
Saturday, May 08, 2010
Jetty 503 - Service Unavailable
This is the type of programming issue that could drive one insane.
For some reason, suddenly getting 503 error in Jetty running Jetty Maven Plugin in Eclipse. That's it. No other explanation. Servlet could not be initialized. No additional clues to be had. Logging is printing to screen. Nada of use.
Finally just recomplied the whole multi-node project. Got errors that certain directories could not be deleted. Could not delete the directories manually until I closed Eclipse and everything related to this thing.
Once I deleted the directories and re-installed all the nodes, it worked again.
For some reason, suddenly getting 503 error in Jetty running Jetty Maven Plugin in Eclipse. That's it. No other explanation. Servlet could not be initialized. No additional clues to be had. Logging is printing to screen. Nada of use.
Finally just recomplied the whole multi-node project. Got errors that certain directories could not be deleted. Could not delete the directories manually until I closed Eclipse and everything related to this thing.
Once I deleted the directories and re-installed all the nodes, it worked again.
Thursday, May 06, 2010
error reading /root/.m2/repository/javax/jms/jms/1.1/jms-1.1.jar; error in opening zip file
So I'm working away minding my own business and suddenly code that has been compiling stops compiling without any changes to the configuration. Apparently Maven has decided it needs to download something different - or re-download something which has moved and suddenly generating errors.
Anyway the error:
error reading /root/.m2/repository/javax/jms/jms/1.1/jms-1.1.jar; error in opening zip file
Maven attempting to download missing files
I agree.
But anyway I figured out that log4J was the problem and removed it from my project since working on different logging mechanism. Why in the world has this been compiling for weeks and suddenly stops? Hmmm?
Additionally after I pulled out log4J I got a bunch of errors about java mail components missing. Apparently those classes must have been part of the log4j jar but I didn't look.
The implication being that my app was using the log4j mailer classes rather than the java mail jar I downloaded from Sun java web site in my web app.
I would look into this further but since I removed log4j from my app my work here is done.
Anyway the error:
error reading /root/.m2/repository/javax/jms/jms/1.1/jms-1.1.jar; error in opening zip file
Maven attempting to download missing files
I agree.
But anyway I figured out that log4J was the problem and removed it from my project since working on different logging mechanism. Why in the world has this been compiling for weeks and suddenly stops? Hmmm?
Additionally after I pulled out log4J I got a bunch of errors about java mail components missing. Apparently those classes must have been part of the log4j jar but I didn't look.
The implication being that my app was using the log4j mailer classes rather than the java mail jar I downloaded from Sun java web site in my web app.
I would look into this further but since I removed log4j from my app my work here is done.
Wednesday, May 05, 2010
Asynchronous HTTP Requests
Informative article on Asynchronous HTTP requests.
Asynchronous HTTP Requests
Ah...but custom protocol handlers...ugh.
Checking out Jetty asynchronous HTTP client...
Jetty Http Client - Asynchronous web requests
Another implementation for review...but this an asycnhronous page fetcher... Hmm....the point of asynchronous is that you don't wait for a response right? At least that was my goal...
Asynchronous Web Page Fetcher
Asynchronous HTTP Requests
Ah...but custom protocol handlers...ugh.
Checking out Jetty asynchronous HTTP client...
Jetty Http Client - Asynchronous web requests
Another implementation for review...but this an asycnhronous page fetcher... Hmm....the point of asynchronous is that you don't wait for a response right? At least that was my goal...
Asynchronous Web Page Fetcher
Cold Fusion Number Precision Problem
From some tax documentation I am reading for Sales Tax Online web tax calculation component:
"You may encounter a precision problem for very large amounts when using the ColdFusion number type. Because this type has a very large exponential range, it necessarily sacrifices precision in the number of significant digits it can carry. This will not generally be a problem with amounts expressed in currencies that have only two significant fractional digits (e.g., US Dollars), however, foreign currencies can have as many as four significant fractional digits. A value with a large non-fractional portion could suffer a rounding error in the fractional digits."
"You may encounter a precision problem for very large amounts when using the ColdFusion number type. Because this type has a very large exponential range, it necessarily sacrifices precision in the number of significant digits it can carry. This will not generally be a problem with amounts expressed in currencies that have only two significant fractional digits (e.g., US Dollars), however, foreign currencies can have as many as four significant fractional digits. A value with a large non-fractional portion could suffer a rounding error in the fractional digits."
Monday, May 03, 2010
Chinese Jar Missing
Hmm. What is this all about.
Problem processing jar entry
com/ibm/icu/impl/data/LocaleElements_zh__PINYIN.class
Suddenly started appearing when running Jetty plugin without me adding anything to my project in terms of dependencies, included classes, etc.
Also coincidentally the whole structure of my Eclipse menus changed as well. Not sure if related. I didn't do anything to update Eclipse. I assume this is all automagic - not sure how I feel about that.
Anyway apparently this error can be ignored (don't usually like that answer) because some jar scanner couldn't find a jar, but didn't stop execution.
Read about it here:
http://www-01.ibm.com/support/docview.wss?uid=swg1PK81656
This is the stack trace:
2010-05-03 23:49:59.202:WARN::Problem processing jar entry com/ibm/icu/impl/data/LocaleElements_zh__PINYIN.class
java.lang.ArrayIndexOutOfBoundsException: 48188
at org.objectweb.asm.ClassReader.readClass(Unknown Source)
at org.objectweb.asm.ClassReader.accept(Unknown Source)
at org.objectweb.asm.ClassReader.accept(Unknown Source)
at org.eclipse.jetty.annotations.AnnotationParser.scanClass(AnnotationParser.java:596)
at org.eclipse.jetty.annotations.AnnotationParser.access$000(AnnotationParser.java:43)
at org.eclipse.jetty.annotations.AnnotationParser$2.processEntry(AnnotationParser.java:576)
at org.eclipse.jetty.webapp.JarScanner.matched(JarScanner.java:152)
at org.eclipse.jetty.util.PatternMatcher.matchPatterns(PatternMatcher.java:82)
at org.eclipse.jetty.util.PatternMatcher.match(PatternMatcher.java:64)
at org.eclipse.jetty.webapp.JarScanner.scan(JarScanner.java:75)
at org.eclipse.jetty.annotations.AnnotationParser.parse(AnnotationParser.java:588)
at org.eclipse.jetty.annotations.AbstractConfiguration.parseWebInfLib(AbstractConfiguration.java:108)
at org.eclipse.jetty.annotations.AnnotationConfiguration.configure(AnnotationConfiguration.java:77)
at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:975)
at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:608)
at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:155)
at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:349)
at org.mortbay.jetty.plugin.JettyWebAppContext.doStart(JettyWebAppContext.java:102)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:55)
at org.eclipse.jetty.server.handler.HandlerCollection.doStart(HandlerCollection.java:165)
at org.eclipse.jetty.server.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:162)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:55)
at org.eclipse.jetty.server.handler.HandlerCollection.doStart(HandlerCollection.java:165)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:55)
at org.eclipse.jetty.server.handler.HandlerWrapper.doStart(HandlerWrapper.java:92)
at org.eclipse.jetty.server.Server.doStart(Server.java:231)
at org.mortbay.jetty.plugin.JettyServer.doStart(JettyServer.java:69)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:55)
at org.mortbay.jetty.plugin.AbstractJettyMojo.startJetty(AbstractJettyMojo.java:433)
at org.mortbay.jetty.plugin.AbstractJettyMojo.execute(AbstractJettyMojo.java:377)
at org.mortbay.jetty.plugin.JettyRunMojo.execute(JettyRunMojo.java:546)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:105)
at org.apache.maven.lifecycle.DefaultLifecycleExecutor.execute(DefaultLifecycleExecutor.java:577)
at org.apache.maven.lifecycle.DefaultLifecycleExecutor.execute(DefaultLifecycleExecutor.java:324)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:247)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:104)
at org.apache.maven.cli.MavenCli.execute(MavenCli.java:427)
at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:157)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:121)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:290)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:230)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:409)
at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:352)
Problem processing jar entry
com/ibm/icu/impl/data/LocaleElements_zh__PINYIN.class
Suddenly started appearing when running Jetty plugin without me adding anything to my project in terms of dependencies, included classes, etc.
Also coincidentally the whole structure of my Eclipse menus changed as well. Not sure if related. I didn't do anything to update Eclipse. I assume this is all automagic - not sure how I feel about that.
Anyway apparently this error can be ignored (don't usually like that answer) because some jar scanner couldn't find a jar, but didn't stop execution.
Read about it here:
http://www-01.ibm.com/support/docview.wss?uid=swg1PK81656
This is the stack trace:
2010-05-03 23:49:59.202:WARN::Problem processing jar entry com/ibm/icu/impl/data/LocaleElements_zh__PINYIN.class
java.lang.ArrayIndexOutOfBoundsException: 48188
at org.objectweb.asm.ClassReader.readClass(Unknown Source)
at org.objectweb.asm.ClassReader.accept(Unknown Source)
at org.objectweb.asm.ClassReader.accept(Unknown Source)
at org.eclipse.jetty.annotations.AnnotationParser.scanClass(AnnotationParser.java:596)
at org.eclipse.jetty.annotations.AnnotationParser.access$000(AnnotationParser.java:43)
at org.eclipse.jetty.annotations.AnnotationParser$2.processEntry(AnnotationParser.java:576)
at org.eclipse.jetty.webapp.JarScanner.matched(JarScanner.java:152)
at org.eclipse.jetty.util.PatternMatcher.matchPatterns(PatternMatcher.java:82)
at org.eclipse.jetty.util.PatternMatcher.match(PatternMatcher.java:64)
at org.eclipse.jetty.webapp.JarScanner.scan(JarScanner.java:75)
at org.eclipse.jetty.annotations.AnnotationParser.parse(AnnotationParser.java:588)
at org.eclipse.jetty.annotations.AbstractConfiguration.parseWebInfLib(AbstractConfiguration.java:108)
at org.eclipse.jetty.annotations.AnnotationConfiguration.configure(AnnotationConfiguration.java:77)
at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:975)
at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:608)
at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:155)
at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:349)
at org.mortbay.jetty.plugin.JettyWebAppContext.doStart(JettyWebAppContext.java:102)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:55)
at org.eclipse.jetty.server.handler.HandlerCollection.doStart(HandlerCollection.java:165)
at org.eclipse.jetty.server.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:162)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:55)
at org.eclipse.jetty.server.handler.HandlerCollection.doStart(HandlerCollection.java:165)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:55)
at org.eclipse.jetty.server.handler.HandlerWrapper.doStart(HandlerWrapper.java:92)
at org.eclipse.jetty.server.Server.doStart(Server.java:231)
at org.mortbay.jetty.plugin.JettyServer.doStart(JettyServer.java:69)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:55)
at org.mortbay.jetty.plugin.AbstractJettyMojo.startJetty(AbstractJettyMojo.java:433)
at org.mortbay.jetty.plugin.AbstractJettyMojo.execute(AbstractJettyMojo.java:377)
at org.mortbay.jetty.plugin.JettyRunMojo.execute(JettyRunMojo.java:546)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:105)
at org.apache.maven.lifecycle.DefaultLifecycleExecutor.execute(DefaultLifecycleExecutor.java:577)
at org.apache.maven.lifecycle.DefaultLifecycleExecutor.execute(DefaultLifecycleExecutor.java:324)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:247)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:104)
at org.apache.maven.cli.MavenCli.execute(MavenCli.java:427)
at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:157)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:121)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:290)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:230)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:409)
at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:352)
Friday, April 30, 2010
Referencing Jar from Maven not in local Repository
If you want to use a jar but not install it in your local Maven repository this seems to work:
Set scope to system. Then point to the jar in systemPath element and pretty much make up the rest.
Set scope to system. Then point to the jar in systemPath element and pretty much make up the rest.
<dependency>
<groupId>com.bmetrix</groupId>
<artifactId>yeehaw</artifactId>
<version>0.1.0</version>
<scope>system</scope>
<systemPath>C:\some\crazy.jar</systemPath>
</dependency>
Saturday, April 24, 2010
JDBC Connection Pool - Minimalist Options
Revisiting database connection management. I took a look at C3P0 which is an open source JDBC driver and looks pretty simple to use.
The only thing I don't like about downloading some of these libraries is the amount of code contained in them I don't really need. I looked at the source and don't need anything for JBoss, for instance. I prefer simplicity and minimalist code, personally. Just to get some functionality in Eclipse I have to download fifty bazillion classes. Yes, bazillion.
And have some other personal reasons for not wanting to use all the J2EE stuff for managing resources. It's my code, so I don't have to conform to the masses in this case. When developing for another company I would probably have to follow Java law.
So anyway I found this interesting example as a starting point for a JDBC connection pool without all the extraneous stuff I don't need:
JDBC Connection Pool - Example Code
I'm sure this needs revisions and not completely what I need (already found it did not override a bunch of things now in the connection interface and is using out of data classes per some documentation I ran across) but maybe something between the two would be nice.
Maybe if components you download had a way to specify and get only the functionality you need when you get them instead of downloading every option under the Sun...no pun intended ;-)
The only thing I don't like about downloading some of these libraries is the amount of code contained in them I don't really need. I looked at the source and don't need anything for JBoss, for instance. I prefer simplicity and minimalist code, personally. Just to get some functionality in Eclipse I have to download fifty bazillion classes. Yes, bazillion.
And have some other personal reasons for not wanting to use all the J2EE stuff for managing resources. It's my code, so I don't have to conform to the masses in this case. When developing for another company I would probably have to follow Java law.
So anyway I found this interesting example as a starting point for a JDBC connection pool without all the extraneous stuff I don't need:
JDBC Connection Pool - Example Code
I'm sure this needs revisions and not completely what I need (already found it did not override a bunch of things now in the connection interface and is using out of data classes per some documentation I ran across) but maybe something between the two would be nice.
Maybe if components you download had a way to specify and get only the functionality you need when you get them instead of downloading every option under the Sun...no pun intended ;-)
Tuesday, April 20, 2010
Use Maven Jetty Plugin in Eclipse
I wanted to run Jetty in Eclipse so I can write code, debug, test, etc. Ran into some issues. Solution here.
Add Jetty plugin to your pom file:
I then tried to run the jetty goal from the command line to keep it simple:
mvn jetty:run
No user specific settings.xml file
Then, something was griping because I never added a settings.xml file to my local repository. Copy the settings.xml file from the directory where you installed maven and drop it into your local repository (see my post on Java development environment if you need more info).
Missing Dependencies
Next...complaints about missing jars not in Maven central repository.
Download the jars I found searching in Google (what would we do without Google as nothing Java related seems to be well documented completely from end to end):
el-impl-2.2.jar
http://download.java.net/maven/2/org/glassfish/web/el-impl/2.2/
el-api-2.2.jar
http://download.java.net/maven/2/javax/el/el-api/2.2/
jsp-impl-2.2.jar
http://download.java.net/maven/2/org/glassfish/web/jsp-impl/2.2/
jsp-api-2.2.jar
http://download.java.net/maven/2/javax/servlet/jsp/jsp-api/2.2/
Install into maven repository with the following commands (make sure you're in the directory where you downloaded the jars or provide full path to jars):
mvn install:install-file -DgroupId=org.glassfish.web -DartifactId=el-impl -Dversion=2.2 -Dpackaging=jar -Dfile=el-impl-2.2.jar
mvn install:install-file -DgroupId=javax.el -DartifactId=el-api -Dversion=2.2 -Dpackaging=jar -Dfile=el-api-2.2.jar
mvn install:install-file -DgroupId=org.glassfish.web -DartifactId=jsp-impl -Dversion=2.2 -Dpackaging=jar -Dfile=jsp-impl-2.2.jar
mvn install:install-file -DgroupId=javax.servlet.jsp -DartifactId=jsp-api -Dversion=2.2 -Dpackaging=jar -Dfile=jsp-api-2.2.jar
PluginsGroup
Next I had to add this to my settings.xml file in my local respository settings.xml file:
<pluginGroups>
<pluginGroup>org.mortbay.jetty</pluginGroup>
</pluginGroups>
No webapp directory or start up file
I noticed that Jetty was referencing a non-existant path for the web app. Default is under your project /src folder in /webapp directory. I also added the default index page under my project src directory:
/src/webapp/index.html
Bug: Jetty context fails with spaces in path
Ok the next issue was that apparently if you have spaces in the path to your local repository Jetty will fail for that reason also.
Details are here: http://jira.codehaus.org/browse/JETTY-1166
So...I had to move my whole local repository to an alternate directory with no spaces. I uncommented the local repository setting in the settings.xml file and changed the path to the new location.
<!-- localRepository
| The path to the local repository maven will use to store artifacts.
|
| Default: ~/.m2/repository-->
<localRepository>/path/to/local/repo</localRepository>
Finally...Jetty runs. If you got this far test it out by going to this url on your local machine:
http://localhost:8080
Running from within Eclipse
Okay that's nice but I wanted to run Jetty from within Eclipse. I right clicked on project pom.xml file, chose "Run As" then "Run Configurations". Double clicked on "Maven Build" node to add a new Maven run configuration. In the dialog box I typed a name - called it "run jetty". I entered the path to the directory containing my pom.xml file as the base directory. For goals I entered "jetty:run". I checked Debug Output, update snapshots and resolve workspace dependencies - clicked Apply, then run.
Yippee. Jetty is running in Eclipse and you can see the output in the Eclipse console.
Add Jetty plugin to your pom file:
<build>
<plugins>
<plugin>
<groupId>org.mortbay.jetty</groupId>
<artifactId>jetty-maven-plugin</artifactId>
</plugin>
</plugins>
<builds>
I then tried to run the jetty goal from the command line to keep it simple:
mvn jetty:run
No user specific settings.xml file
Then, something was griping because I never added a settings.xml file to my local repository. Copy the settings.xml file from the directory where you installed maven and drop it into your local repository (see my post on Java development environment if you need more info).
Missing Dependencies
Next...complaints about missing jars not in Maven central repository.
Download the jars I found searching in Google (what would we do without Google as nothing Java related seems to be well documented completely from end to end):
el-impl-2.2.jar
http://download.java.net/maven/2/org/glassfish/web/el-impl/2.2/
el-api-2.2.jar
http://download.java.net/maven/2/javax/el/el-api/2.2/
jsp-impl-2.2.jar
http://download.java.net/maven/2/org/glassfish/web/jsp-impl/2.2/
jsp-api-2.2.jar
http://download.java.net/maven/2/javax/servlet/jsp/jsp-api/2.2/
Install into maven repository with the following commands (make sure you're in the directory where you downloaded the jars or provide full path to jars):
mvn install:install-file -DgroupId=org.glassfish.web -DartifactId=el-impl -Dversion=2.2 -Dpackaging=jar -Dfile=el-impl-2.2.jar
mvn install:install-file -DgroupId=javax.el -DartifactId=el-api -Dversion=2.2 -Dpackaging=jar -Dfile=el-api-2.2.jar
mvn install:install-file -DgroupId=org.glassfish.web -DartifactId=jsp-impl -Dversion=2.2 -Dpackaging=jar -Dfile=jsp-impl-2.2.jar
mvn install:install-file -DgroupId=javax.servlet.jsp -DartifactId=jsp-api -Dversion=2.2 -Dpackaging=jar -Dfile=jsp-api-2.2.jar
PluginsGroup
Next I had to add this to my settings.xml file in my local respository settings.xml file:
<pluginGroups>
<pluginGroup>org.mortbay.jetty</pluginGroup>
</pluginGroups>
No webapp directory or start up file
I noticed that Jetty was referencing a non-existant path for the web app. Default is under your project /src folder in /webapp directory. I also added the default index page under my project src directory:
/src/webapp/index.html
Bug: Jetty context fails with spaces in path
Ok the next issue was that apparently if you have spaces in the path to your local repository Jetty will fail for that reason also.
Details are here: http://jira.codehaus.org/browse/JETTY-1166
So...I had to move my whole local repository to an alternate directory with no spaces. I uncommented the local repository setting in the settings.xml file and changed the path to the new location.
<!-- localRepository
| The path to the local repository maven will use to store artifacts.
|
| Default: ~/.m2/repository-->
<localRepository>/path/to/local/repo</localRepository>
Finally...Jetty runs. If you got this far test it out by going to this url on your local machine:
http://localhost:8080
Running from within Eclipse
Okay that's nice but I wanted to run Jetty from within Eclipse. I right clicked on project pom.xml file, chose "Run As" then "Run Configurations". Double clicked on "Maven Build" node to add a new Maven run configuration. In the dialog box I typed a name - called it "run jetty". I entered the path to the directory containing my pom.xml file as the base directory. For goals I entered "jetty:run". I checked Debug Output, update snapshots and resolve workspace dependencies - clicked Apply, then run.
Yippee. Jetty is running in Eclipse and you can see the output in the Eclipse console.
SQL Server Permissions
Had an interesting issue after restoring a database to SQL Server 2008 database server. None of the users could select from any tables. When restoring a SQL Server database, you pretty much always have to go in and recreate the accounts and add them to the databases again with the appropriate permissions. This was a bit different and odd.
Here's how I solved it:
1. Tried adding a select role which had select permissions on the database (did not solve it but here's now to do it):
Let's say your user name is "dbuser" and you want to create some roles to do different things like execute, insert, select, etc. You could do something like this - which is probably overkill and you might want to set up a different schema also - but I was just testing anyway:
/* CREATE ROLES */
CREATE ROLE db_execute
CREATE ROLE db_select
CREATE ROLE db_delete
CREATE ROLE db_insert
CREATE ROLE db_update
/* GRANT PERMISSIONS TO ROLES */
GRANT EXECUTE ON SCHEMA :: dbo TO db_execute
GRANT SELECT ON SCHEMA :: dbo TO db_select
GRANT DELETE ON SCHEMA :: dbo TO db_delete
GRANT INSERT ON SCHEMA :: dbo TO db_insert
GRANT UPDATE ON SCHEMA :: dbo TO db_update
/* ADD USER TO ROLES */
EXEC sp_addrolemember 'db_execute', 'dbuser'
EXEC sp_addrolemember 'db_update', 'dbuser'
EXEC sp_addrolemember 'db_insert', 'dbuser'
EXEC sp_addrolemember 'db_delete', 'dbuser'
EXEC sp_addrolemember 'db_select', 'dbuser'
2. You could grant permission to all tables and views in your database like this handy dandy script I found here
use MYDBNAME
DECLARE @login varchar(50)
SET @login = 'dbuser'
DECLARE @tables TABLE(ROWID int IDENTITY(1,1), SQLSTR varchar(500))
INSERT INTO @tables
SELECT 'GRANT SELECT ON ' + NAME + ' TO '+@login
FROM sysobjects
WHERE TYPE = 'U'
AND NAME NOT LIKE 'SYNC%'
DECLARE @rowid int, @sqlstr varchar(500)
SET @rowid = 0
SET @sqlstr = ''
DECLARE grant_tbl_cursor CURSOR FOR
SELECT ROWID, SQLSTR
FROM @tables
ORDER BY ROWID
OPEN grant_tbl_cursor
FETCH NEXT FROM grant_tbl_cursor
INTO @rowid,@sqlstr
WHILE @@FETCH_STATUS = 0
BEGIN
EXECUTE (@sqlstr)
FETCH NEXT FROM grant_tbl_cursor
INTO @rowid,@sqlstr
END
CLOSE grant_tbl_cursor
DEALLOCATE grant_tbl_cursor
DECLARE @views TABLE(ROWID int IDENTITY(1,1), SQLSTR varchar(500))
INSERT INTO @views
SELECT 'GRANT SELECT ON ' + NAME + ' TO '+@login
FROM sysobjects
WHERE TYPE = 'V'
SET @rowid = 0
SET @sqlstr = ''
DECLARE grant_vw_cursor CURSOR FOR
SELECT ROWID, SQLSTR
FROM @views
ORDER BY ROWID
OPEN grant_vw_cursor
FETCH NEXT FROM grant_vw_cursor
INTO @rowid,@sqlstr
WHILE @@FETCH_STATUS = 0
BEGIN
EXECUTE (@sqlstr)
FETCH NEXT FROM grant_vw_cursor
INTO @rowid,@sqlstr
END
3. But what really solved the problem for me was looking at the effective permissions on the table that I initially tested. Right click, choose properties and check out the effective permissions tab. I noticed that the public role was denied pretty much any permission on any database object. Given that all users must be in the public role and cannot be removed, had to fix that.
First I verified that these permissions were also returning from a query on the permissions and users for that object:
select OBJECT_NAME(major_id) as object,
USER_NAME(grantee_principal_id) as grantee,
USER_NAME(grantor_principal_id) as grantor,
Permission_name,
state_desc, class
from sys.database_permissions
where major_id=OBJECT_ID('Table_In_Question')
Sure enough, public had deny set on just about everything.
So...I undenied it but removing all permissions from the public role and resetting permissions as desired.
REVOKE SELECT FROM PUBLIC
It's a little odd that we are revoking the denying of something, but that's how it works. Running my permissions query after this showed that the public user had no object level select priveledges on that table - neither grant nor deny. I tested and was able to select from the table with the other user that was part of the public role.
I also tested after my changes and that same user was not able to change permissions or table strucures (which is what I was after in the first place via more granular database permissions.) Further review of system tables and probably schema changes are in order, but this particular problem is solved.
Here's how I solved it:
1. Tried adding a select role which had select permissions on the database (did not solve it but here's now to do it):
Let's say your user name is "dbuser" and you want to create some roles to do different things like execute, insert, select, etc. You could do something like this - which is probably overkill and you might want to set up a different schema also - but I was just testing anyway:
/* CREATE ROLES */
CREATE ROLE db_execute
CREATE ROLE db_select
CREATE ROLE db_delete
CREATE ROLE db_insert
CREATE ROLE db_update
/* GRANT PERMISSIONS TO ROLES */
GRANT EXECUTE ON SCHEMA :: dbo TO db_execute
GRANT SELECT ON SCHEMA :: dbo TO db_select
GRANT DELETE ON SCHEMA :: dbo TO db_delete
GRANT INSERT ON SCHEMA :: dbo TO db_insert
GRANT UPDATE ON SCHEMA :: dbo TO db_update
/* ADD USER TO ROLES */
EXEC sp_addrolemember 'db_execute', 'dbuser'
EXEC sp_addrolemember 'db_update', 'dbuser'
EXEC sp_addrolemember 'db_insert', 'dbuser'
EXEC sp_addrolemember 'db_delete', 'dbuser'
EXEC sp_addrolemember 'db_select', 'dbuser'
2. You could grant permission to all tables and views in your database like this handy dandy script I found here
use MYDBNAME
DECLARE @login varchar(50)
SET @login = 'dbuser'
DECLARE @tables TABLE(ROWID int IDENTITY(1,1), SQLSTR varchar(500))
INSERT INTO @tables
SELECT 'GRANT SELECT ON ' + NAME + ' TO '+@login
FROM sysobjects
WHERE TYPE = 'U'
AND NAME NOT LIKE 'SYNC%'
DECLARE @rowid int, @sqlstr varchar(500)
SET @rowid = 0
SET @sqlstr = ''
DECLARE grant_tbl_cursor CURSOR FOR
SELECT ROWID, SQLSTR
FROM @tables
ORDER BY ROWID
OPEN grant_tbl_cursor
FETCH NEXT FROM grant_tbl_cursor
INTO @rowid,@sqlstr
WHILE @@FETCH_STATUS = 0
BEGIN
EXECUTE (@sqlstr)
FETCH NEXT FROM grant_tbl_cursor
INTO @rowid,@sqlstr
END
CLOSE grant_tbl_cursor
DEALLOCATE grant_tbl_cursor
DECLARE @views TABLE(ROWID int IDENTITY(1,1), SQLSTR varchar(500))
INSERT INTO @views
SELECT 'GRANT SELECT ON ' + NAME + ' TO '+@login
FROM sysobjects
WHERE TYPE = 'V'
SET @rowid = 0
SET @sqlstr = ''
DECLARE grant_vw_cursor CURSOR FOR
SELECT ROWID, SQLSTR
FROM @views
ORDER BY ROWID
OPEN grant_vw_cursor
FETCH NEXT FROM grant_vw_cursor
INTO @rowid,@sqlstr
WHILE @@FETCH_STATUS = 0
BEGIN
EXECUTE (@sqlstr)
FETCH NEXT FROM grant_vw_cursor
INTO @rowid,@sqlstr
END
3. But what really solved the problem for me was looking at the effective permissions on the table that I initially tested. Right click, choose properties and check out the effective permissions tab. I noticed that the public role was denied pretty much any permission on any database object. Given that all users must be in the public role and cannot be removed, had to fix that.
First I verified that these permissions were also returning from a query on the permissions and users for that object:
select OBJECT_NAME(major_id) as object,
USER_NAME(grantee_principal_id) as grantee,
USER_NAME(grantor_principal_id) as grantor,
Permission_name,
state_desc, class
from sys.database_permissions
where major_id=OBJECT_ID('Table_In_Question')
Sure enough, public had deny set on just about everything.
So...I undenied it but removing all permissions from the public role and resetting permissions as desired.
REVOKE SELECT FROM PUBLIC
It's a little odd that we are revoking the denying of something, but that's how it works. Running my permissions query after this showed that the public user had no object level select priveledges on that table - neither grant nor deny. I tested and was able to select from the table with the other user that was part of the public role.
I also tested after my changes and that same user was not able to change permissions or table strucures (which is what I was after in the first place via more granular database permissions.) Further review of system tables and probably schema changes are in order, but this particular problem is solved.
Monday, April 19, 2010
Run Batch Files with Cold Fusion
For some processes that use batch files to execute scripts related to processing builds, database imports, data syncs, etc. We also want to allow users who don't have access to the server remotely via remote desktop to be able to execute these commands but also log who executed them when and send out notifications. By surrounding these processes with a secure UI we can allow users to execute these functions without giving remote desktop access to the server, require authentication and track who executed what, when. ColdFusion happens to be the language available in this environment, and this is the ColdFusion tag that runs a batch file.
<cfexecute
name="ApplicationName"
arguments="CommandLine Arguments"
OUTPUTfile="Output file name"
timeout="Timeout interval in seconds">
With code such as this will be important to ensure that only the specified scripts can be executed by limiting user accounts that can run this and against what files and directories.
<cfexecute
name="ApplicationName"
arguments="CommandLine Arguments"
OUTPUTfile="Output file name"
timeout="Timeout interval in seconds">
With code such as this will be important to ensure that only the specified scripts can be executed by limiting user accounts that can run this and against what files and directories.
Friday, April 16, 2010
Backup and Restore SVN Repository
Basic command for backing up SVN repository:
svnadmin dump repositorypath > backupname.svn
More on backups at:
Back up SVN Repository
To restore the backup on another machine:
svnadmin load respositorypath < backupname.svn
svnadmin dump repositorypath > backupname.svn
More on backups at:
Back up SVN Repository
To restore the backup on another machine:
svnadmin load respositorypath < backupname.svn
Thursday, April 15, 2010
Truncate SQL Server Transaction Log
USE DatabaseName
GO
DBCC SHRINKFILE(<TransactionLogName>, 1)
BACKUP LOG <DatabaseName> WITH TRUNCATE_ONLY
DBCC SHRINKFILE(<TransactionLogName>, 1)
GO
Transaction logs are truncated on backup or checkpoints depending on the model you're using:
http://technet.microsoft.com/en-us/library/ms189085.aspx
However the backup will include the larger transaction log so you'll want to back up after truncation to get a smaller backup file and not have to restore the entire transaction log.
GO
DBCC SHRINKFILE(<TransactionLogName>, 1)
BACKUP LOG <DatabaseName> WITH TRUNCATE_ONLY
DBCC SHRINKFILE(<TransactionLogName>, 1)
GO
Transaction logs are truncated on backup or checkpoints depending on the model you're using:
http://technet.microsoft.com/en-us/library/ms189085.aspx
However the backup will include the larger transaction log so you'll want to back up after truncation to get a smaller backup file and not have to restore the entire transaction log.
Subscribe to:
Posts (Atom)