Search This Blog

Monday, October 3, 2011

Openworld 10/3 Oracle Keynote

I think its interesting that Oracle has allowed so much time to their partner/competitor EMC and VMWare in the Monday morning keynote.  The value of EMC's big data strategies, Greenplum, VMWare and EMC Storage w/vfast are all being shown to Oracle's best customers.

They're talking about "Cloud meets big data"...and showing they can get 1 million iops with vfast storage in VMWare..like Exadata, while making available all the virtualization features we've become used to, like snap/clone.

Pared with Greenplum and hadoop, peyabytes are analyzed in seconds.  It'll be interesting to hear Exahadoop's and Exalytics response. :)

Here's a pic of an Exalytics server:


Friday, September 30, 2011

Free at Last! (to post about EHCC on ZFS!)

About a month ago, I made a post re:HCC working on ZFS...which I later had to remove, because it hadn't officially been announced yet.  Finally, it has been officially announced to the world:
 
http://www.oracle.com/us/corporate/press/508020

When I originally posted about this, I was told that it was due to new functionality being put into the firmware of the Sun 7420 that would allow it...that HCC compression would be offloaded to the CPU of the head of the 7420.  Kevin Closson correctly pointed out in a comment on my post that it wasn't a new feature in the 7420...there isn't HCC "offloading" like in a storage cell...its just that the code that prevents this from working is being removed if you're using storage from Oracle (Sun/Pillar Axiom).  Kevin went on to say the feature of HCC on non-storage cell storage made it through beta testing...there was no technical reason to prevent the functionality.

He was clearly correct, because the official announcement states, "Hybrid Columnar Compression...is enabled in an update to Oracle Database 11g Release 2."  ...no mention of a firmware update.  That'll be the last time I listen to a non-technical source about a new technical feature.  I'm just glad I didn't try to argue with Kevin. :)

About a year ago I was in an RFP with different vendors to provide a database solution to a client.  The client went with Exadata...the primary reason was that HCC would reduce the amount of storage needed.  Without HCC, much more storage would have to be purchased, and that cost more than offset the additional cost of licensing incurred by the Exadata platform.  It was a huge selling point that the other storage vendors couldn't match.  I hear rumors that may change in the future, however.... ;)

Anyway, I can see why Oracle would make that an Oracle-storage-only feature, I witnessed it making them a multi-million dollar deal.  In a perfect world, Oracle would allow the feature to work on all storage platforms, and if people chose to go with the Pillar Axiom or Sun storage, it would be on the ample merits of their storage platforms alone.  The customer would be allowed to choose what they believe to be the best storage solution to match their database, all other things (and compression features) being equal.  So...making this an Oracl-only feature is good for Oracle, but possibly bad for the customer.

I wonder how different this "new" feature is than what we first saw in patch 8896202 (for 11.2.0.1 on OEL/RH).  This is the patch to enable the compression advisor to estimate Exadata HCC compression ratios.  It showed that HCC worked on any storage without Exadata.  I first used that patch when it came out...about a year and a half ago, on a linux vm with vmdk storage.  The thing is, when you do a trace...its not just an estimate...it literally takes your data and compresses it with HCC, and gives you the results.  Soooo...I guess I'm just reinforcing the point that this isn't a new technical breakthrough, HCC on non-Exadata is a feature that's no longer prevented (as long as you buy the storage from Oracle).


It does move in the right direction for the customer though...7420 storage is much cheaper than a storage cell.  I look forward to trying it out.  Maybe I'll post some performance numbers of HCC on the 7420 vs a storage cell.  Hmmmmm........

Openworld 2011 (How to fill out your sessions)

I always put off signing up for my sessions...if you're the same way, and you're new to Openworld, maybe this will help you.

Sessions are often disappointing because...from the description you expect a "deep dive" into the technology, and they often end up being a marketing presentation, not at all technical.  Last year I was at a session whose subject included the phrase "deep dive"...and it was still very light.  I hate that...because there are always options for the session, and you don't know if you made the wrong choice until at least a few minutes in.  At that point...it seems rude to get up and leave and try to find a better session...but do it anyway...this is your valuable time, don't waste it.

Another common pitfall to sessions that, from their description look like they'll be interesting, is when a non-technical manager gets up and speaks to the project, without knowing the technical details.  These are usually a waste of time, but not always, because usually his techy underlings are lurking in the audience, and at the end of the session, during question/answer, he'll defer all the questions to the techies.  At that point you start getting valuable information.

So, my strategy this year is to primarily pick the sessions based on the presenters.  Any of these presentations will be excellent:CLICK HERE  also HERE. After that, I populate the schedule with the database path, and then I go back and use search terms to find things like: "hadoop", "storage", "Exadata", "Backup", "Database Appliance", "RAC" etc.  Then go back and prioritize the ones that have time conflicts.

When you try to sign up for a session that's at the same time as a different session and you get a time conflict, it asks you if you want to move the "other" session to "your interests"...say yes...because that way if you're in a session that tricked you into thinking it was going to be interesting...and it turns out to either be a high level marketing session or a manager talking about what his techy team accomplished (I hate those) then while you're sitting there you can pull out your Droid/IPhone and use the Openworld 2011 app to see what your alternate session that gave you a time conflict was.  If it isn't filled up, you might be able to leave and go there...hopefully it'll be better.

I wish there was some kind of a rating scale for the sessions to give you guidance on how technical they are...but even if they had it, it would be subjective, and you'd still have the same problem.  I think if you follow the guidelines above you'll be in reasonably good shape.

Rumors of Openworld 2011

Its that time of year when everybody starts guessing re:What Larry will have to say for us that's new and exciting from Openworld.  Here are some guesses:

1. 11.2.0.3 is already out for Linux from metalink/MOS...following the new pattern of 11.2.0.2 where its a complete download rather than a patchset.  So...with that out of the way...I've been seeing patches "Backported from 12.1" for months now on MOS.  Its a good bet Larry is going to drop RDBMS 12 on us.

2. We've all heard by now about Oracle's new database appliance.  Its a stepping stone for smaller organizations to Exadata.  A managed database/hardware solution with good performance.  I plan to check it out at some point later this year...I'll let you know how that goes.  Today I got an email from Oracle that mentions "Oracle Database Appliance 12c Private Database Cloud."  So...this lends credence to #1 above, and...interestingly it'll be called...not 12i, not 12g...but 12c.

3.  Without giving details this time (I was asked to remove the post talking about a change to HCC as it relates to storage), let me just say...there will be a big, positive change to HCC as it relates to storage.

4. This one is huge:  Oracle's Hadoop appliance!  I heard about this a few months ago, but I can finally talk about it.  There have been tweets and blogs mentioning "Exa-Hadoop", and here's something official that confirms it:<>.  I think this is an interesting play for Oracle...Hadoop's profitability in an appliance is clear...they make money on the hardware and on support.  Since the product is Opensource, they'll help the entire Hadoop community with their development (Like OVM helps Xen and OEL and OCFS2 helps linux.) There's huge interest in the industry around Hadoop.

5. Exalogic v2!  I've heard from a somewhat unreliable source (you know who you are) that Exalogic will now have better support for Cache Fusion...this will be a huge performance boost.

6. OEM 12 (or maybe 12c?)!  I don't know if this will be announced, but the timing seems really good.  See my previous post re:my friend who was encouraged by Oracle to remove his post re:the coming features of OEM 12c.  :)   There's some pretty cool stuff on the way...huge changes.  I saw a mock up of the new UI about a year ago...its completely different and vastly improved. 

I just read a funny post <> on this topic...Oracle really should come out with a product called "SexiaBIHadoop."

There are always surprises...can't wait to be there. :)

Friday, August 19, 2011

Page Missing?

A close friend of mine recently blogged about an upcoming feature of an Oracle product...let's call it OEM.  So, in his zeal to discuss the latest, greatest and the new technology in OEM, when a product manager sent him an email and asked about how he knew about the new features, my friend removed his post.  Was that a wise move or did it display a lack of courage?  Who's to say...maybe both?   The real downside is that there was an interesting conversation between him and an Oak Table member I was enjoying. The lessons we should learn from this are, "Secret information isn't always declared so."  The other lesson is..."He who has the most lawyers wins arguments before they begin."

Monday, August 15, 2011

Equi-sized Partition Cuts

There are a lot of resources to show you how to make a partitioned table...syntax search results are plentiful...but its unusual to find a resource that tells you about the process to partition a table.

Some rows are wider than others, and a lot of times your partition key doesn't have gaps...so this may not apply to you.  When partitioning my approach has been to:

1. Identify the access patterns
    You can do this by talking to the application development teams and selecting from hist_sql_plan (see below).  That will give you a good idea about some of the sql that's hitting the table in question.  Is it direct-io reads or conventional dml?  Are they simple queries, where response time is likely important, or are they more complex, taking several minutes to complete?  Look at the operation column.

select * from dba_hist_sql_plan where sql_id in (select sql_id from DBA_HIST_SQL_PLAN where object_name='TABLE_NAME') order by sql_id, id;

2. From #1, what columns do you often see in the where clauses?  This/these columns may be good partition key column candidates.

3. From #1, what's the best compression type you can use without negatively affecting application performance?  (long runtime queries could likely have more aggressive compression on their tables, short queries will have to be C4OLTP or no compression)

4.  If possible, use interval partitioning instead of range partitioning.  There are a lot of things that will keep you from using interval partitioning  (like domain indexes), but it'll save you time in management if you can use it.


5.  There's a balance between adding lots of partitioning for performance reasons and having too many partitions to be easily managed.  As a rule of thumb, I shoot for ~20, depending on table size, but never let a partition be smaller than 2% of the db_cache size (see earlier post re:smallness logic).  This is hard with ASMM/AMM, since the db_cache size is dynamic...but even if you use ASMM and AMM you should set minimum sizes in the memory pools.  Its a good practice to make the partitions as equally sized as possible...if that means equal number of rows or equally size segments, that's for you to decide...it depends on your situation.  Also consider the partitioned tables commonly joined with this table...if your queries pull back many rows, partition-wise joins will make you want to use the same partitioning column if possible.


We're partitioning many, many tables and we need a process to do this as quickly as possible.  Some of the tables are multi-terrabye, and it was taking me a really long time to come up with the values for the ranges.  So, I changed my method and just created a query to do it.  There might be a better way...but this is performing very well, even on multi-billion row tables this returns in around a minute.  If you have a better method, I'd love to hear about it.

The idea here is that there are repeated values and/or gaps in the sequence column you want to make your partitioning key...so you can't just take the sequence number.  The first partition has to be the row that's 5% in...regardless of what the value of the partitioned key is.  This query will give you the value of the partitoning column key that's about 1/20th of the rows in the table...the 2nd will give you the one that's around 2/20th's...etc.  Change the table_name to be your table, and seq_id to be your partition key.


select part, max(seq_id), max(rownumber) from (
select trunc(rownumber,2) part, tm2.* from (
select /*+ PARALLEL 96 */ seq_id, CUME_DIST() OVER (PARTITION BY 'X' ORDER BY seq_id) AS RowNumber
from table_name tn
) tm2
where trunc(rownumber,2) in (0.05,0.1,0.15,0.2,0.25,0.3,0.35,0.4,0.45,0.5,
0.55,0.6,0.65,0.7,0.75,0.8,0.85,0.9,0.95,1))
group by part
order by 1;

...when its done, it'll give you 3 columns.  The first one is your "goal" (.6 is the one ~60% into the row count), the second is the partition key value at that point, the 3rd is how close to your goal you came (which depends on the data cardinality...how unique the column values are.)  Use the 2nd column as the value for your partitioned table range. IE:

0.05    20      0.06
0.1      36      0.11
0.15    52      0.16
0.2      67      0.21
0.25    82      0.26
0.3      95      0.31
0.35    124    0.36
0.4      151    0.41
0.45    169    0.46
0.5      186    0.51
0.55    200    0.56
0.6      217    0.61
0.65    235    0.66
0.7      253    0.71
0.75    296    0.76
0.8      311    0.81
0.85    347    0.86
0.9      381    0.91
0.95    415    0.96
1         445    1

PARTITION BY RANGE (SEQ_ID)

  PARTITION P_1 VALUES LESS THAN (20),
  PARTITION P_2 VALUES LESS THAN (36),
  PARTITION P_3 VALUES LESS THAN (52),
  PARTITION P_4 VALUES LESS THAN (67),
  PARTITION P_5 VALUES LESS THAN (82),
  PARTITION P_6 VALUES LESS THAN (95),
  PARTITION P_7 VALUES LESS THAN (124),
  PARTITION P_8 VALUES LESS THAN (151),
  PARTITION P_9 VALUES LESS THAN (169),
  PARTITION P_10 VALUES LESS THAN (186),
  PARTITION P_11 VALUES LESS THAN (200),
  PARTITION P_12 VALUES LESS THAN (217),
  PARTITION P_13 VALUES LESS THAN (235),
  PARTITION P_14 VALUES LESS THAN (253),
  PARTITION P_15 VALUES LESS THAN (296),
  PARTITION P_16 VALUES LESS THAN (311),
  PARTITION P_17 VALUES LESS THAN (347),
  PARTITION P_18 VALUES LESS THAN (381),
  PARTITION P_19 VALUES LESS THAN (415),
  PARTITION P_20 VALUES LESS THAN (445),
  PARTITION P_MAX VALUES LESS THAN (MAXVALUE)
)

I hope this helps you, or at least saves you some time determining your table partition key and the best range of values for that key.

Partitoning while migrating to Exadata-P2 (Exadata is inexpensive)

Preliminary testing results are very promising.  After a few days of activity the accelerated performance due to C4OLTP is working, and the majority of the data is seeing excellent compression (up to 55X+) for HCC c4archive-high. 

One of the tables is ~2TB in size and contains a clob column.  If you read the documentation, it tells you that HCC doesn't work on cLOBS.  What they SHOULD say is it doesn't work on out-of-line clobs.  If your clobs are small, they're stored in the table's segment and HCC does work on them.  I was able to take this 2048GB table and compress it down to around 76GB.  That's 3.7% of its original size, for a compression ratio of about 27X.

This is one of the concepts that's difficult for people considering buying Exadata to grasp...it costs a lot, but due to its compression abilities, it may be the least expensive thing out there for very large databases.  Its hard to compare Exadata to a standard 11.2 database system because Hybrid Columnar Compression makes it an apples-oranges comparison.

Consider the cost of high-performance storage on an EMC or Hitachi array.  Everybody has their own TCO for high performance storage...but lets say EMC gives you a good deal and it costs $15/GB.  The storage savings on this table alone saved almost $30k.  Multiply that out by the 28TB in a high performance machine and it saves $11.3 million.  A different way to look at it is...in a world where you can compress everything with HCC-QH and get these compression results...the actual capacity of a 28TB high capacity Exadata machine is 756TB!

Now that Oracle is selling storage cells individually, Exadata is truly expandable...there really isn't a capacity limit until you can saturate IB and create a bottleneck...but since that's been optimized (only sending required blocks, columns and sometimes result sets to the RAC nodes), its difficult to image how much storage you can have before that's an issue.