As 2014 draws to a close I working on finishing off a number of tasks and projects. One of these tasks is an annual one for me. The task is to list all the things I’ve done as an Oracle ACE Director. If has been a very busy year, not just with ACE activities but also work wise too. That will explain why I have been a bit quiet on the blogging side of things in recent months.
In 2014 I one major highlight. It was the publication of my book Predictive Analytics using Oracle Data Miner by Oracle Press. Many thanks for everyone involved in writing this book, especially my family and the people in Oracle Press who gave me the opportunity.
Here is my summary.
January : BIWA Summit : 2 presentations (San Francisco, USA) **
March : OUG Ireland Conference (Dublin, Ireland)
April : OUG Norway : 2 presentations (Oslo, Norway) **
June : OUG Finland : 2 presentations (Helsinki, Finland) **
June : Oracle EMEA Data Warehousing Global Leaders Forum (Dublin, Ireland)
August : OUG Panama : 2 presentations (Panama City, Panama) **
August : OUG Costa Rica : 3 presentations (San Jose, Costa Rica) **
August : OUG Mexico : 2 presentations (Mexico City, Mexico) **
September : Oracle Open World (San Francisco, USA) **
December : UKOUG Tech15 : 2 presentations (Liverpool, UK)
December : UKOUG Apps15 (Liverpool, UK)
That is 19 hours of presenting this year.
** Many thanks to the Oracle ACE Director programme for funding the flights and hotels for these conferences. All other expenses and conferences I paid for out of my own pocket.
My ODM Book
On the 8th August my book titled Predictive Analytics using Oracle Data Miner, was published by Oracle Press. It all began 12 months and 2 weeks previously. I had the book written and the technical edits done by the middle of February (2014). Between March and June the Copy edits and layouts where completed. The book is ideal for any data scientist, Oracle developer, Oracle architect and Oracle DBA, who want to use the in-databse data mining functionality. That way they can use and build upon their existing SQL and PL/SQL skills to perform predictive analytics.
The book is available on Amazon and comes in Print and eBook formats
Oracle Open World
This year I got to go to my second ACE Director briefing. This is held on the Thursday and Friday before OOW. At the briefing we get lots of Very senior people coming in to tell us what is happening with the products in their area and what the plans are over the next 12 to 18 months. Lots of what we are told is all under NDA. My favourite part of this briefing is when Thomas Kurian comes in and talks for about 90 minutes. No slides, no notes. The first 15 minutes is him telling us what Larry & Co are going to announce at OOW, that are the main product directions etc. Then he opens to the floor for questions. You can ask him anything about the set of Oracle products (>3000) and he will explain in detail what is happen. He even commented on the plans for the Oracle Games Console this year!!!
This year I had the opportunity to present at OOW again. It was a joint presentation with Roel Hartman and we had the pleasure of being one of the first presentations at OOW at 8:30am on the Sunday morning. Despite the early start we have really good turn out for our presentation.
Then I got to enjoy OOW with all the various activities, presentations, entertainment and hanging out at the OTN lounge with the other ACEs and ACEDs.
One of the things I really like doing is playing with various features of Oracle and then writing some blog posts about them. Most of what I blog about evolves around the SQL & PL/SQL Statistics functions and the Advanced Analytics Option, comprising Oracle Data Mining and Oracle R Enterprise. In addition to these blog posts I also have posts relating to various Oracle User Group activities. So there is a good mixture of material on the blog.
In 2014 I have written 60 blog posts (including this one). This number is a little be less than previous years and perhaps the main reason for that is due to me being extremely busy with various project work this year.
OTN has accepted three articles from me in 2014. I was delighted about these acceptances and I’m looking forward to writing some more articles in 2015 for them.
- Sentiment Analysis using Oracle Data Miner
- ROracle : How to get Started and Commands you need to Know
- Predictive Queries in 12c
I have a few more ideas for articles and I will be writing these in 2015. We will have to wait and see if OTN will accept them.
My Oracle Magazine Collection & Reviews
You may or may not be aware that I’ve been collecting Oracle Magazine for over 20 years now. I have nearly the entire collection of Oracle Magazine going back to the very first edition. Check out the collection here. You will see that I’m missing a few and these are highlighted by the grey boxes. If you do have any of these and you would like to donate to my collection then please get in touch.
One of the things I like to blog about is on some of these old Oracle Magazines. If you go to my Oracle Magazine collection page you will see the past editions that I have writing a review of. Click on the links to view the blog post review an edition.
In 2014 I have written reviews of the following:
- Oracle Magazine-July/August 2000
- Oracle Magazine September/October 2000
- Oracle Magazine-November/December 2000
- Oracle Magazine-January/February 2001
The Oracle User Group in Ireland (OUG Ireland) has continued to grow this year in membership but also with the number of attendees at our events. In March of each year we have our flagship event which is our annual conference. This year we have almost 300 people and unfortunately people had to turned away at the door because we had headed the maximum limit on the number of attendees for the event. Planing has already commenced for 2015 and the call for presentations is now open. Hopefully 2015 will be bigger and better that 2014. We had a second day at the conference this year where we had Tom Kyte give a full day seminar. Again this was fully booked out for weeks/months before hand. In March 2015 we will be having a second day of the conference with Maria Colgan giving a one day workshop/seminar on the In-Memory option and the Optimiser. You cannot book your place on this seminar yet but then it does open make sure to book your place quick as I’m sure it will book out very quickly.
We also had a number of TECH and BI SIGs and the number of attendees has significantly grown over the past few years. This is fantastic and hopefully this will continue. If it does then maybe we might be able to put on more SIG events.
In the editor of Oracle Scene Magazine which is published by the UKOUG. This was my first full year as editor after spending many years as deputy editor. In 2014 we have published three editions of Oracle Scene and I would like to thanks everyone who has submitted an article. You have helped grow the quality of the contents and also grow the readership numbers. The calls for articles for the Spring edition is now open.
My Oracle Data Science newsletter & My Oracle User Group Weekly newsletter
A couple of years ago I set up a news aggregator based on Twitter feeds and on updates from certain websites. I’ve divided these into two different newsletters. The first is My Oracle Data Science News and as you might guess it is focused on the worlds of Data Science, Predictive Analytics and related developments with a bit of a focus towards the Oracle world. This newsletter gets published each day.
My second newsletter is focused on Oracle User Group activities around the World and is again based on the various Twitter handles of the Oracle User Groups. I’m include over 40 OUG Twitter handles in the aggregator so I should be picking up almost everyone. If you discover your OUG is not being included then drop me an email and I’ll add you to the list. This newsletter goes out every Friday.
Plans for 2015 so far
The start of 2015 is already very busy and I’m already booked for 3 conferences BIWA Summit (CA, USA), OUG Norway and OUG Ireland.
Planing for OUG Ireland is under ways and we are hoping to build on the successes we have had over the past few years.
So as editor of Oracle Scene magazine we are planning for our first issue in 2015, the call for articles is open and we have been busy recruiting authors of articles on specifics.
I’m sure I’ve forgotten a few things, I usually do.
It has been a fun year. I’ve made lots of friends around the World and I look forward to meeting you all at some conference in 2015.
If you are downloading the EA1 of SQL Developer that includes Oracle Data Miner (ODMr), and you intend to use Oracle Data Miner then you will need to update the ODMr Repository.
You could do it the hard way and run the upgrade repository sql scripts that are located in the …\sqldeveloper-220.127.116.11.29-no-jre\sqldeveloper\dataminer\scripts directory.
Or you could do it the easy way and let the inbuilt functionality in Oracle Data Miner do it for you.
To do it the easy way all you need to do is to open the ODMr Connections window and the double click on one of your ODM connections.
ODMr will check the version of the repository you have installed and if needed it will prompt you about upgrading the repository. Select Yes and you will be prompted to enter the SYS password. So talk kindly with your DBA for them to enter the password for you. Then click on the Start button. They will lick off the OMDr Repository Upgrade scripts.
NB: Make sure you have a backup of your workflows before you do this. A little think happened to me during the SQL Dev / ODMr 4.0 upgrade back in September 2013 where all my workflows disappeared. You can imagine how happy I was about that. Since then the ODMr team have added some functionality to ensure something like this doesn’t happen again. But you never know.
To backup your ODMr workflows use the Export Workflow option.
When the repository upgrade has finished you will get a ‘Task Complete Successfully’ message in the upgrade window. Click on the close button and away you go with this updated version.
A few days ago the first Early Adaptor release of SQL Developer 4.1 (EA1) was made available. You can go ahead and download it from here and make sure to check out the blog post by Jeff Smith on some install and setup that is required around the latest version of Java.
I’ve been using SQL Developer since its very first release, so getting my hands on a new release is very exciting. There are lots and lots of new features in the tool. Again check out the blog posts by Jeff Smith and Kris Rice on some of these new features. I really like the new DBA screens 🙂 But this screen really needs some scroll bars and not everything fits on my screen. So Jeff and Kris if you are reading this, can you add some scroll bars.
In addition they have been working on “new” SQL*Plus that is called SDSQL. This is a new command line tool that is supposed to be bigger and better than SQL*Plus but still gives us a command line tool to run our scripts and demos. To download and install the tool go to here.
As you know I’m a bit of an Oracle Data Miner/Mining fan. There are now new in-database features, but there are a lot of new features in the GUI tool (aka ODMr) along with some improvements and bug fixes. Here is a list of the ODMr 4.1 EA1 new and updated features (taken from the ODMr Help in SQL Dev)
JSON Data Support for Oracle Database 18.104.22.168 and above
In response to the growing popularity of JSON data and its use in Big Data configurations, Data Miner now provides an easy to use JSON Query node. The JSON Query node allows you to select and aggregate JSON data without entering any SQL commands. The JSON Query node opens up using all of the existing Data Miner features with JSON data. The enhancements include:
Data Source Node
Automatically identifies columns containing JSON data by identifying those with the IS_JSON constraint.
Generates JSON schema for any selected column that contain JSON data.
Imports a JSON schema for a given column.
JSON schema viewer.
Create Table Node
Ability to select a column to be typed as JSON.
Generates JSON schema in the same manner as the Data Source node.
JSON Data Type
Columns can be specifically typed as JSON data.
JSON Query Node
Ability to utilize any of the selection and aggregation features without having to enter SQL commands.
Ability to select data from a graphical layout of the JSON schema, making data selection as easy as it is with scalar relational data columns.
Ability to partially select JSON data as standard relational scalar data while leaving other parts of the same JSON document as JSON data.
Ability to aggregate JSON data in combination with relational data. Includes the Sub-Group By option, used to generate nested data that can be passed into mining model build nodes.
Improved database session management resulting in less database sessions being generated and a more responsive user interface.
Filter Columns Node
Combined primary Editor and associated advanced panel to improve usability.
Explore Data Node
Allows multiple row selection to provide group chart display.
Classification Build Node
Automatically filters out rows where the Target column contains NULLs or all Spaces. Also, issues a warning to user but continues with Model build.
Enhanced workflows to ensure that Loading, Reloading, Stopping, Saving operations no longer block the UI.
Revised the Online Help to adhere to topic-based framework.
Selected Bug Fixes (does not include 4.0 patch release fixes)
GLM Model Algorithm Settings: Added GLM feature identification sampling option (Oracle Database 12.1 and above).
Filter Rows Node: Custom Expression Editor not showing all possible available columns.
WebEx Display Issues: Fixed problems affecting the display of the Data Miner UI through WebEx conferencing.
Denny Wong of the ODM team in Oracle has made available a tutorial on importing JSON data for use with ODMr. Check it out here.
I’ve been told there will be a couple of tutorials on the new features coming out (from the ODMr team) over the next few weeks. So keep an eye out of these.
Check out my blog post on what you need to do to get started/using ODMr 4.1 EA1.
The UKOUG annual conferences commence on Sunday 7th December and run until Wednesday 10th.
Like previous years there are two conferences, one called TECH15 and the other is called APPS15. You might guess what each conference is about!!.
This year these conferences are being held at the same time and in the same venue. But they are separate conferences!.
This year I’ve been very lucky (or very unlucky) to have 3 presentations at these conferences. Two of these will on part of the TECH15 conference and one will be part of the APPS15 conference.
Just in case you interested in what I’m presenting about and you might want to attend them, here is the list with the room numbers.
10:30-11:20 : Oracle Advanced Analytics in Oracle Fusion Apps & Beyond (Apps) (Room : Ex1)
11:30-12:20 : Predictive Queries in Oracle 12c (TECH) (Room : Hall 6)
11:30-12:20 : What are they thinking? With APEX and Oracle Data Miner. (TECH) (Room : Ex4)
(this is a joint presentation with Roel Hartman)
Yes on the Monday I have 2 back-to-back presentation with a 10 minute gap to get from one side of the conference centre to the other side 😦 I’m not looking forward to that transition, but I’m sure it will be fine.
OUG Ireland Call for submissions is now open.
The closing date for submissions is 5th January, 2015.
and the submission webpage can be found here.
The OUG Ireland conference will be on Thursday 19th March. Yes it is only a one day conference 😦 but we will be 5 or 6 or more streams. So there will be something for everyone and plenty of choice.
On Friday 20th March we will have Maria Colgan, formally the Optimizer Lady and now the In-Memory Queen (or something like that), giving a full day workshop on the In-Database option and the Optimizer. She will also be about for the main conference on the 19th, so you can expect a presentation or two from her on the Thursday.
Agenda selection day is the 8th January, 2015. So hopefully you will be getting the acceptance emails soon after that or during week of 12th January.
There is a committee of about 10 people who are involved in selecting presentations and setting the agenda. If it was up to me then I would accept everything/everyone. So if your presentation is not accepted this time, please don’t blame me 🙂 I said YES to your presentation, I really, really did. I fought so hard to have your presentation included. If your presentation is not accepted then the blame is down to the other committee members 🙂
The conference will be held in Croke Park, and is a 15-20 minute taxi ride from the Airport.
You can follow the Conference and other OUG Ireland events using the twitter tag #oug_ire
With the release of the Oracle Database 22.214.171.124 there was a number of new features and options. Most of the publicity has been around the in-Memory option. But there was lots of other features for the DBA and a few for the developer.
One of the new SQL functions is the APPROX_COUNT_DISTINCT(). This function is different to the tradition count distinct, COUNT(DISTINCT expression), in that is performs an approximate count distinct. The theory is that this approximate count is a lot more efficient than performing the full count distinct.
The APPROX_COUNT_DISTINCT() function is really only suitable when you are processing very large volumes of data and when the data set contains a large number of distinct values.
The general syntax of the function is:
… APPROX_COUNT_DISTINCT(expression) …
and returns a Number.
The function returns the approximate number of records that contain distinct value for the expression.
The APPROX_COUNT_DISTINCT() function ignores records that contain a null value for the expression. Plus is performs less work on the sorting and aggregations. Just run and Explain Plan and you can see the differences.
In some of the material from Oracle the APPROX_COUNT_DISTINCT() function can be 5x to 50x++ times faster. But it depends on the number of distinct values and the complexity of the SQL query.
As the result / returned value from the function may not be 100% accurate, Oracle says that the functions has an accuracy of >97% (with 95% confidence).
The function cannot be used on the following data types: BFILE, BLOB, CLOB, LONG, LONG RAW and NCLOB
When Oracle Data Miner (ODMr) 4.0 (which is part of SQL Developer) came out back in late 2013 there was a number of new features added to the tool. One of these was a Graph node that allows us to create various graphs and charts that include Line, Scatter, Bar, Histogram and Box plot.
I’ve been using this node recently to produce graphs and particularly scatter plots. I’ve been using the scatter plots to graph the Actual values in a data set against the Predicted values that was generated by ODMr. In this scenario I had a separate data set for training my ODM data mining models and another testing data set for, well testing how well the model performed against an unseen data set.
In general the graphs produced by the Graph node look good and gives you the information that you need. But what I found was that as you increased the size of the data set, the scatter plot can look a messy. This was in part due to the size of the square used to represent a data point. As the volume of data increased then your scatter plot could just look like a coloured in area of blue squares. This is illustrated in the following image.
What I discovered today is that you can zoom in on this graph to explore different regions and data point on it. This do this you need to select an data that is within the x-axis and y-axis area. When you do this you will see a box form on your graph that selects the area that you indicate by moving your mouse. After you have finished selecting the area, the Graph Node will zooms into this part of the graph and shows the data points. For example if I select the area from about 1000 on the x-axis and 1000 on the y-axis, I will get the following.
Again if I select a similar are area of 350 on the x-axis and 400 on the y-axis I get the following zoomed area.
You can keep zooming in on various areas.
At some point you will have finished zooming in and you will want to return to the original graph. To zoom back outward all you need to do in the graph is to click on it. When you do this you will go back to the previous step or image of the graph. You can keep doing this until you get back to the original graph. Alternatively you can zoom in and out on various parts of the graph.
Hopefully you will find this feature useful.