Oracle + Open Source: Oracle Open Source Projects

« Open Source Driving Patent Reform | Main | BarCampNYC - Good Stuff »

January 13, 2006

Oracle Open Source Projects - The Interviews

Back in 2000 I recall searching for Open Source projects to cover in the book I wrote Oracle and Open Source. My co-author and I found seven web-based applications, a few with Perl and Tcl, four Java tools, and five GTK applications. A search for the keyword "Oracle" on freshmeat a popular Open Source project indexing page, now yields an incredible 184 projects. Granted some may be libraries, or supporting components, but its a heck of a lot of activity.

This story is really about tinkerers, folks that like to play with technologies, and offer up their creations to everyone. Some
end up with serious projects on their hands, and a large following. When Oracle ported to Linux, it was a moment for the tinkerers to get busy.

We've managed to contact the authors of six Oracle Open Source projects, and ask them a few questions about their projects, and how they got started. The authors include:

Ljubomir Buturovic, author of gqlplus
Itzchak Rehberg, author of OraRep
Jeff Horwitz, author of extproc_perl
Paul Vallee, the author of m2o.pl
Tim Strehle, author of the OracleEditor
Clausen Yngve, author of ora2html

In 1998 Oracle released it's Enterprise database for Linux. Truth be told, there were a few of us out there trying to get Oracle 7 SCO binaries working under a Linux emulator and had some success. Nevertheless rumors were running around on the Oracle DBA email lists that inside Oracle, porting had already been finished, and it was just a matter of time. So October of 1998 was an exciting moment, for it meant that tinkerers could start really getting their hands dirty with Oracle. And that they did. Around that time I started an Open Source project called Karma solely to monitor Oracle databases. It got some attention but further, drew me into Open Source development, where before I had just been an enthusiast who utilized the tools and applications everyday. Then in 2000 I started to collaborate on a book for O'Reilly entitled Oracle and Open Source. This was released in April 2001, and brought my attention to the growing world of Open Source applications and tools for Oracle.

With that introduction, here are the six questions we asked each of the authors.

1. Some people got their start with Open Source at a University, doing development with gcc, or using other GNU tools, and naturally gravited to Linux when the project began. When did you first get involved with open source software, and what were your first experiences?

2. When did you first decide to start your own open source project and what motivated you most?

3. In your own words, what does you application, tool or library do? What are it's primary or outstanding features?

4. How did you get involved with Oracle? Were you using the Oracle database before Oracle ported to Linux? As a developer or DBA?

5. Where do you see your development efforts with Oracle moving in the future? What would you like to see happen to your project?

6. What might you like to see Oracle doing more of to encourage development of Open Source applications?

Ljubomir Buturovic, author of gqlplus answers the six questions as follows:

1. I first started using Emacs and gcc in 1992, doing research at Boston University. Later on, as a UNIX developer in 1995, a colleague told me about a free UNIX variant you could run on your own home computer. That's how I got involved with Linux.

2. I wrote this tool (gqlplus) because I needed it to improve my work productivity. Then I decided I should release it for two reasons:

- it is a relatively simple utility which would not be too difficult to maintain
- a couple of colleagues told me that they like it and that other Oracle users may benefit from it

3. In a word, gqlplus is sqlplus with command-line editing, name completion and command history. As most Oracle users know, sqlplus is somewhat inconvenient when it comes to repeating past commands, correcting typing mistakes, issuing slightly different queries etc. gqlplus alleviates this problem by incorporating SQL command editing similar to UNIX shell (bash/tcsh). In addition, it also has table-name and column-name completion, so you don't have to remember long or cryptic names. And it's a drop-in replacement for sqlplus, meaning (almost) everything works exactly the same as sqlplus, except that there is the new added functionality.

4. I've been using Oracle on various UNIX system since 1996 as a developer. I have only recently - 2004 - tried running Oracle on Linux.

5. I would like to see Oracle incorporate this functionality in sqlplus, which would make my project obsolete :-) Not that I don't want to maintain it, but there are some functions which can only be supported by changing sqlplus internally, so that's what I would prefer Oracle to do.

6. I would suggest continue improving support for a wider range of Linux distributions, for example Fedora. Right now Oracle installation on Fedora is decidedly non-trivial exercise which may discourage some potential users

Itzchak Rehberg, author of OraRep had this to say:

1/2. That is already about 10 years ago, and I was running OS/2 at that time. I missed some little utilities, so I decided to write them on my own. Since I was using the script language Rexx for this, these utilities became Open Source as a side effect.

After having started developing my own tools this way, I found Open Source very useful: I could take ideas and even routines from other projects easily without any "permission" or license problems. I was using other Open Source programs, and this way could give back something to the community. So I decided to take the GPL for all of my developments, as far as possible.

Meanwhile I switched to Linux some years ago, and mainly develop in PHP. I ran and run multiple projects for different purposes, all of them are using the GPL. A list of programs/scripts and more details on them you can find at:
- http://www.qumran.org/homes/izzy/ (my private and personal site)
- http://www.izzysoft.de/?topic=software (my business site)

3. Since I had to answer this question for more than one or two applications, please refer to the links quoted for question 1.

4. I was working for a company about 5 years ago, mainly as a developer for PHP based web applications. Since the company was running not only MySQL (I was already used to), but also Oracle databases, I got involved into that. First by writing PHP applications that accessed Oracle databases, together with the problem setting up PHP accordingly. Later on, I became the Oracle DBA for the company by running through the Oracle certification program (becoming an OCP DBA). That was after Oracle was ported to Linux: While the production databases where running under Solaris 8, some of our test databases have been set up under Linux.

5. Not easy to answer globally. For me personally, I would like to see an easier integration of Oracle into PHP, e.g. by providing an appropriate module that would be easy to plug in. At the moment, it is quite tricky to set it up, and you can't do that easily with an RPM paket or the like but rather have to re-compile everything from the scratch. If the PHP setup for the Oracle connections would be as easy as copy a few files and add a line to the php.ini (i.e. setting up an extension), I would provide Oracle support to more of my applications, which right now only support MySQL and PostgreSQL, which belong to the "shipped" PHP contributions.

6. See above. PHP got quite common for web development. Oracle databases are quite common in commercial environments. To bring both together, and encourage developers to support Oracle databases with their PHP/database driven applications, an easy integration of Oracle into PHP would help a lot!

These answers are from Jeff Horwitz, author of extproc_perl.

1. That's exactly where I got started. I was a student programmer at the University of Michigan when I was first exposed to open source software. At first I was limited to just downloading, porting, and compiling various packages, but in time I began writing my own software. In 1997 I wrote my first open source package, Authen::Krb4, which provides a Perl API to Kerberos 4.

2. Each project is different, but I assume you want to talk about extproc_perl, which is the only Oracle project on my plate. As a system administrator, I work very closely with our DBA. Knowing my experience with Perl, he commented to me one day, "Gee Jeff, wouldn't it be nice if I could use regular expressions in my select statement?" Now, this was back in 2001, long before 10g's regular expression support. I took that as a personal challenge, and that night, extproc_perl was born.

3. extproc_perl embeds a Perl interpeter in an Oracle external procedure, allowing you to write stored procedures in Perl. Among other things, it supports per-session interpreter persistence, the ability to query/update the database within the existing transaction, and automatic type conversion between Oracle and Perl. The most intriguing feature though, is the ability to leverage the vast number of modules in CPAN (the official Perl module archive) for your stored procedures.

4. I've been a system administrator at a university and two companies that all happened to use Oracle. I think the first time I actually developed anything for it was back in 1997, well before the Linux port. But I'm still just a sysadmin, and I rely on my friendly DBA for most database tasks. For someone who has done what I've done with Oracle, I know surprisingly little about it!

5. As I'm involved with several open source projects, my involvement on each of them, including extproc_perl, tends to ebb and flow like the tides. Development stalled on extproc_perl in mid-2004 as I began work on a new project, but I expect to get back into the swing of things for another release in 2005.

6. Get the word out. Publicize projects that use Oracle in an open source environment. Most people are simply not aware of the quality open source projects that are available to them, and rely on vendors like Oracle to tell them how to do X, Y, and Z. That's all well and good for what Oracle does best, but as we like to say in the Perl world, there's more than one way to do it.

We interview Paul Vallee, the author of m2o.pl, and here's what he had to say.

1. Well, at Pythian we've been running Oracle on Linux for our customers since around 1999, so our experience with open source software started primarily as a beneficiary. We really love the cost savings and flexibility it provides, not to mention the dramatic pace of feature improvements all along the stack.

2. When we first wrote m2o.pl, it was in support of a single migration and interfacing project for a client. Our contract with the client had us owning the intellectual property we built, and yet we had no anticipated use for the software once the project was over. What a waste! So we decided to open-source it because we felt that that would allow the work to be reused in other shops.

3. Well, m2o.pl is a very simple tool that does a very simple thing: it takes mysql create syntax and parses it, then outputs it back into Oracle-compatible create syntax.

4. I first worked with Oracle when it was Oracle 5 on... believe it or not... PC-DOS! I was a developer tasked with a statistical macrosimulation analysis of defense human resources data and we had Oracle 5 and SAS to choose from. I'm embarassed now to admit that for this purpose, SAS beat out Oracle handily. However, since then I've continuously worked with Oracle primarily as a DBA and DBA manager. In 1997, I co-founded Pythian Remote DBA and to this day our primary support focus is the Oracle database.

5. Honestly, I am amazed at how much interest the project receives and how many hits I get on our website directly related to m2o. However, if there's one bone of contention or disappointment is the ratio of feature quibbles to patches I get - Most of the feedback I receive sounds a lot more like "your software didn't handle this correctly" and not a lot like "here's a patch that fixes this Oracle 10 issue for you". I believe that the average user of open-source software has become less and less willing to contribute back over the years.

6. Oracle could open-source their legacyware. For instance, Oracle advanced replication has been completely rewritten since the Oracle 7 days when it was delivered in wrapped PL/SQL. However - that replication totally worked! No doubt their new replication is better - so why haven't they open-sourced the Oracle 7 PL/SQL yet? We could be riffing off that and building fancy new things overlaid on top of it. Replication is just one example of software Oracle has shelved instead of releasing. It would be trivial to come up with more.

The following answers are from Tim Strehle, author of the OracleEditor tool.

1. Back in university, and during my first day job, I got in touch with some commercial Unix flavors - coming from Windows (where I developed Microsoft Access applications), this was a scary world.

So I knew there would be a lot to learn when I started working for Digital Collections. Their first programs had been developed on NeXT computers; later they moved to a client/server model with Unix servers and Mac and Windows clients, all software being written in C. But at the time I joined them, their next major software release was to be a web application written in PHP, running on Unix, Apache and Oracle.

The only thing I knew about PHP 3 was its website, and simply from browsing the online manual, you could tell that PHP would be a good choice for developing web applications. Even at that time it supported so many databases, had lots of useful-looking extensions - and the best online language and function reference I had seen so far.

I was able to learn PHP incredibly fast, and while the PHP manual contributed to that success, it helped even more that my boss, Thies C. Arntzen, was one of the core PHP developers and an Apache Software Foundation member. He taught me the foundations and the tricks of the language, and it was fascinating to see how a PHP feature request or bug discussed with Thies resulted in a solution implemented in the next official PHP release. I really appreciate the privilege of having learned PHP directly from a PHP genius. It has been a great introduction into the world of open source software...

2. Like probably most software developers, from time to time there's a need to build a tool for in-house use. We wouldn't mind giving these tools away for free, but usually they aren't polished or generic enough to be useful for others.

In early 2003, while building a new in-house tool, I decided to take the time to make it useful for people outside our company as well. While I was curious to find out what kind of feedback I would receive, my main motiviation was that I wanted to give something back: I'm making a living building (closed source) software on open source projects like Linux, Apache and PHP, so it felt right to contribute some free software myself.

3. My company's (commercial, closed source) software was running on Oracle on Solaris, HPUX, Irix and AIX; I was developing on Oracle 8 on Solaris. The advent of Oracle on Linux was very exciting for us, it made it so much easier to run Oracle on relatively cheap hardware.

4. I hope that my OracleEditor.php script will continue to attract new users, and perhaps new co-developers - and I love getting suggestions for improvement.

Oracle has proven to be a solid foundation to build upon, so there's a promising future for Oracle-based tools like mine.

5. For years, you felt like a niche developer when building PHP software on Oracle. This feeling has really changed with the advent of Oracle's Open Source Developer Center, the JDeveloper 10g PHP Extension and the announcement that PHP would be included in Oracle Application Server 10G. That's been very encouraging, and all I wish for is that the Open Source Developer Center stays alive and keeps getting frequent updates. Good work!

6. I'm looking forward to seeing what you're making out of these interviews.

The following answers are from Clausen Yngve, author of ora2html.

1. The first time I laid my hands on anything open source, was around '96, when I attended a course in OS Design at the University of Tromso (In Norway). To provide us with a safe environment for programming (safe as in not ruining things for other people :-), we were provided with laptops running Red Hat linux. The transition from using win* products was eased somewhat by the fact that the univeristy was running UX terminals, and I had been using them for regular work and programming for about a year in advance. But I'm not going to claim the switch was without failures ;-) I was slightly tempted to convert my home-PC to Linux or BSD at the time, but it just seemed a little bit too advanced for me. And I couldn't play Doom and Quake running Linux ;-)

At the time I didn't really give much thought to what was open source or not - and I guess I didn't _really_ start using Linux/GNU/Open Source software seriously until '98 when I started working as a unix sysadmin. From there on I've been using open source software on a regular basis.

2. In early 2002 it started to dawn on me that there were a lot of things I didn't know about configuration, features and possibilities in the oracle database - and I didn't have a clue where in the database to look for information about all the stuff I didn't know about :-) At the same time, I faced the challenge of keeping track of a growing number of databases in my company. Keeping manual track of a handful of databases isn't too bad, but after looking up version and options information for 50 databases, you sort of loose interest ;-)

I eventually decided to create an automated routine for extraction of general database information to help me in my daily work. Inspired by the cfg2html project (Where a few of my friends were involved), I decided to share my utility just in case somebody else would find it useful.

3. My utlity is a small-ish shell script that collects information about configuration and setup of oracle software and databases installed on a server. It logs on to the available databases and runs checks on different aspects of database operation, including memory config, storage utilization, security, installed options, versions ... and so on. The end result is saved as an html file for easy browsing.

It's mainly a utility geared towards DBAs that want an easy method of getting system information. I've found it to be a useful source of reference information in administrative tasks like capacity planning, database upgrade/migration, space management and security auditing. It could probably be useful in disaster/recovery scenarios also.

I recommend running it from cron with automated transfer of the end result to a web server. That makes the info easily accessible. I've also found the source code to be a good place for cannibalizing parts for other DBA-related scripts ;-)

4. I've been working with Oracle since '98 when I started working as a unix sysadm with part-time DBA responsibility. That was on an HP platform. I'm not sure which Oracle version was the first to be ported to linux, but I did try the 8.0 release. I've never done much work on Oracle/Linux combinations, though.

5. My project is really just a small utility that doesn't require much effort to keep updated. These days it's just updated as a result of discovered bugs, or upon requests. There are still 10g features I've omitted from my utility, but until I see a real use for the info, I'll delay it.

6. Hm. I don't think I have any well-founded thoughts on that issue. Yet ;-)

Posted by admin at January 13, 2006 01:20 AM

Trackback Pings

TrackBack URL for this entry:
http://WWW.ORACLEOPENSOURCE.COM/mt/mt-tb.cgi/2

Listed below are links to weblogs that reference Oracle Open Source Projects - The Interviews:

» blog and ping from blog and ping
blest .. niceblog1 [Read More]

Tracked on February 3, 2006 11:18 PM

» blog and ping from blog and ping
blest .. niceblog1 [Read More]

Tracked on February 3, 2006 11:19 PM

» payday loans from payday loans
Niceblog!!! The post was most informative. [Read More]

Tracked on February 7, 2006 09:08 PM

» debt consolidation from debt consolidation
Rubber, debt consolidation which their of. [Read More]

Tracked on February 9, 2006 10:16 PM

» debt consolidation from debt consolidation
Explanation of london the himself. debt consolidation [Read More]

Tracked on February 9, 2006 10:19 PM

» horse cocks from
comics young naked girls teen titans hentai lingerie modeling hot ... [Read More]