Oracle Tips and Tricks — David Fitzjarrell

April 14, 2015

Columns, Columns Everywhere

Filed under: General,joins — dfitzjarrell @ 08:54

"Many of the things which can never be, often are."
- Norton Juster, The Phantom Tollbooth 

Oracle’s most recent database release has an interesting problem involving ANSI joins and tables having 500 or more columns. Yes, this is likely a rare situation in OLTP systems but it exists, nonetheless. [Data warehouses are different and it’s likely that tables of 500 columns or more could exist.] It was brought to light in a forum I frequent. Given the right conditions a query of the form:


select a.*
from a 
join b on ...
join c on ...;

throws an ORA-01792 even though none of the involved tables has 1000 columns defined. Let’s look at an example and see what Oracle is doing with the ANSI join:


SQL> 
SQL> --
SQL> -- Create 500 column tables
SQL> --
SQL> declare v_sql varchar2(32767);
  2  
  3  begin
  4  
  5    -- create test table ORANGE with 50 columns
  6    v_sql := 'create table ORANGE (Z_COL_001 number(9) not null';
  7    for i in 1 .. 499 loop
  8      v_sql := v_sql || ',' || chr(10) || 'Z_COL_' || lpad(to_char(i+1), 3, '0') || ' varchar2(2) null';
  9    end loop;
 10    v_sql := v_sql || ')';
 11    execute immediate v_sql;
 12  
 13    -- create test table MAUVE with 50 columns
 14  
 15    v_sql := 'create table MAUVE (Z_COL_001 number(9) not null';
 16    for i in 1 .. 499 loop
 17      v_sql := v_sql || ',' || chr(10) || 'T_COL__' || lpad(to_char(i+1), 3, '0') || ' varchar2(2) null';
 18    end loop;
 19    v_sql := v_sql || ')';
 20    execute immediate v_sql;
 21  
 22  end;
 23  /

PL/SQL procedure successfully completed.

SQL> 
SQL> -- create test table YELLOW with 2 columns
SQL> 
SQL> create table YELLOW (Z_COL_001 number(9) not null, P_COLUMN_002 varchar2(2));

Table created.

SQL> 
SQL> --********************************************************************************--
SQL> -- selecting one column work!
SQL> --********************************************************************************--
SQL> 
SQL> select p.P_COLUMN_002
  2  from ORANGE g
  3    join MAUVE r on g.Z_COL_001 = r.Z_COL_001
  4    join YELLOW p on g.Z_COL_001 = p.Z_COL_001;

no rows selected

SQL> 
SQL> --********************************************************************************--
SQL> -- notice the YELLOW table only has 2 columns, but selecting p.* fails in Oracle 12c:
SQL> -- SQL Error: ORA-01792: maximum number of columns in a table or view is 1000
SQL> --********************************************************************************--
SQL> 
SQL> select p.*
  2  from ORANGE g
  3    join MAUVE r on g.Z_COL_001 = r.Z_COL_001
  4    join YELLOW p on g.Z_COL_001 = p.Z_COL_001;
  join YELLOW p on g.Z_COL_001 = p.Z_COL_001
       *
ERROR at line 4:
ORA-01792: maximum number of columns in a table or view is 1000 


SQL> 

The query above relies on star expansion of the select list; lest you think that is the source of the problem let’s look at a query where only one column is selected from the join:


SQL> --********************************************************************************--
SQL> -- sub selecting only one column also fails in Oracle 12c:
SQL> -- SQL Error: ORA-01792: maximum number of columns in a table or view is 1000
SQL> --********************************************************************************--
SQL> 
SQL> select (select 1 from dual where 1 = 2) as exp_column
  2  from ORANGE g
  3    join MAUVE r on g.Z_COL_001 = r.Z_COL_001
  4    join YELLOW p on g.Z_COL_001 = p.Z_COL_001;
select (select 1 from dual where 1 = 2) as exp_column
                      *
ERROR at line 1:
ORA-01792: maximum number of columns in a table or view is 1000 


SQL> 
SQL> drop table ORANGE;

Table dropped.

SQL> drop table MAUVE;

Table dropped.

SQL> drop table YELLOW;

Table dropped.

SQL> 

So it isn’t the star expansion causing issues, it appears to be the ANSI join. The original example doesn’t allow for successful execution of the dbms_utility.expand_sql_text procedure so the example is modified to create tables with 499 columns; the error no longer appears but dbms_utility.expand_sql_text reveals an interesting result:


SQL> --
SQL> -- Create 499 column tables so we can see the expansion problem
SQL> --
SQL> declare v_sql varchar2(32767);
  2  
  3  begin
  4  
  5    -- create test table ORANGE with 50 columns
  6    v_sql := 'create table ORANGE (Z_COL_001 number(9) not null';
  7    for i in 1 .. 498 loop
  8      v_sql := v_sql || ',' || chr(10) || 'Z_COL_' || lpad(to_char(i+1), 3, '0') || ' varchar2(2) null';
  9    end loop;
 10    v_sql := v_sql || ')';
 11    execute immediate v_sql;
 12  
 13    -- create test table MAUVE with 50 columns
 14  
 15    v_sql := 'create table MAUVE (Z_COL_001 number(9) not null';
 16    for i in 1 .. 498 loop
 17      v_sql := v_sql || ',' || chr(10) || 'T_COL__' || lpad(to_char(i+1), 3, '0') || ' varchar2(2) null';
 18    end loop;
 19    v_sql := v_sql || ')';
 20    execute immediate v_sql;
 21  
 22  end;
 23  /

PL/SQL procedure successfully completed.

SQL> 
SQL> -- create test table YELLOW with 2 columns
SQL> 
SQL> create table YELLOW (Z_COL_001 number(9) not null, P_COLUMN_002 varchar2(2));

Table created.

SQL> 
SQL> --********************************************************************************--
SQL> -- selecting one column work!
SQL> --********************************************************************************--
SQL> 
SQL> select p.P_COLUMN_002
  2  from ORANGE g
  3    join MAUVE r on g.Z_COL_001 = r.Z_COL_001
  4    join YELLOW p on g.Z_COL_001 = p.Z_COL_001;

no rows selected

SQL> 
SQL> --********************************************************************************--
SQL> -- notice the YELLOW table only has 2 columns, but selecting p.* fails in Oracle 12c:
SQL> -- when the joined tables have more than 499 columns each
SQL> -- SQL Error: ORA-01792: maximum number of columns in a table or view is 1000
SQL> --********************************************************************************--
SQL> 
SQL> select p.*
  2  from ORANGE g
  3    join MAUVE r on g.Z_COL_001 = r.Z_COL_001
  4    join YELLOW p on g.Z_COL_001 = p.Z_COL_001;

no rows selected

SQL> 
SQL> --********************************************************************************--
SQL> -- Here's why:
SQL> --
SQL> -- When Oracle attempts an ANSI expansion of the code columns multiply in the ANSI
SQL> -- join and somehow appear to be associated with the various tables included in
SQL> -- the join
SQL> --********************************************************************************--
SQL> 
SQL> set serveroutput on
SQL> set long 20000
SQL> 
SQL> variable m_sql_out clob
SQL> 
SQL> declare
  2      m_sql_in    clob :=
  3                          '
  4                          select p.*
  5                          from ORANGE g
  6                          join MAUVE r on g.Z_COL_001 = r.Z_COL_001
  7                          join YELLOW p on g.Z_COL_001 = p.Z_COL_001
  8                          ';
  9  begin
 10  
 11      dbms_utility.expand_sql_text(
 12          m_sql_in,
 13          :m_sql_out
 14      );
 15  
 16  end;
 17  /

PL/SQL procedure successfully completed.

SQL> 
SQL> column m_sql_out wrap word
SQL> print m_sql_out
SELECT "A1"."Z_COL_001_998" "Z_COL_001","A1"."P_COLUMN_002_999" "P_COLUMN_002"  
FROM  (SELECT "A3"."Z_COL_001_0" "Z_COL_001","A3"."Z_COL_002_1"                 
"Z_COL_002","A3"."Z_COL_003_2" "Z_COL_003","A3"."Z_COL_004_3"                   
"Z_COL_004","A3"."Z_COL_005_4" "Z_COL_005","A3"."Z_COL_006_5"                   
"Z_COL_006","A3"."Z_COL_007_6" "Z_COL_007","A3"."Z_COL_008_7"                   
"Z_COL_008","A3"."Z_COL_009_8" "Z_COL_009","A3"."Z_COL_010_9"                   
"Z_COL_010","A3"."Z_COL_011_10" "Z_COL_011","A3"."Z_COL_012_11"                 
"Z_COL_012","A3"."Z_COL_013_12" "Z_COL_013","A3"."Z_COL_014_13"                 
"Z_COL_014","A3"."Z_COL_015_14" "Z_COL_015","A3"."Z_COL_016_15"                 
"Z_COL_016","A3"."Z_COL_017_16" "Z_COL_017","A3"."Z_COL_018_17"                 
"Z_COL_018","A3"."Z_COL_019_18" "Z_COL_019","A3"."Z_COL_020_19"                 
"Z_COL_020","A3"."Z_COL_021_20" "Z_COL_021","A3"."Z_COL_022_21"                 
"Z_COL_022","A3"."Z_COL_023_22" "Z_COL_023","A3"."Z_COL_024_23"                 
"Z_COL_024","A3"."Z_COL_025_24" "Z_COL_025","A3"."Z_COL_026_25"                 
"Z_COL_026","A3"."Z_COL_027_26" "Z_COL_027","A3"."Z_COL_028_27"                 
"Z_COL_028","A3"."Z_COL_029_28" "Z_COL_029","A3"."Z_COL_030_29"                 
"Z_COL_030","A3"."Z_COL_031_30" "Z_COL_031","A3"."Z_COL_032_31"                 
"Z_COL_032","A3"."Z_COL_033_32" "Z_COL_033","A3"."Z_COL_034_33"                 
"Z_COL_034","A3"."Z_COL_035_34" "Z_COL_035","A3"."Z_COL_036_35"                 
"Z_COL_036","A3"."Z_COL_037_36" "Z_COL_037","A3"."Z_COL_038_37"                 
"Z_COL_038","A3"."Z_COL_039_38" "Z_COL_039","A3"."Z_COL_040_39"                 
"Z_COL_040","A3"."Z_COL_041_40" "Z_COL_041","A3"."Z_COL_042_41"                 
"Z_COL_042","A3"."Z_COL_043_42" "Z_COL_043","A3"."Z_COL_044_43"                 
"Z_COL_044","A3"."Z_COL_045_44" "Z_COL_045","A3"."Z_COL_046_45"                 
"Z_COL_046","A3"."Z_COL_047_46" "Z_COL_047","A3"."Z_COL_048_47"                 
"Z_COL_048","A3"."Z_COL_049_48" "Z_COL_049","A3"."Z_COL_050_49"                 
"Z_COL_050","A3"."Z_COL_051_50" "Z_COL_051","A3"."Z_COL_052_51"                 
"Z_COL_052","A3"."Z_COL_053_52" "Z_COL_053","A3"."Z_COL_054_53"                 
"Z_COL_054","A3"."Z_COL_055_54" "Z_COL_055","A3"."Z_COL_056_55"                 
"Z_COL_056","A3"."Z_COL_057_56" "Z_COL_057","A3"."Z_COL_058_57"                 
"Z_COL_058","A3"."Z_COL_059_58" "Z_COL_059","A3"."Z_COL_060_59"                 
"Z_COL_060","A3"."Z_COL_061_60" "Z_COL_061","A3"."Z_COL_062_61"                 
"Z_COL_062","A3"."Z_COL_063_62" "Z_COL_063","A3"."Z_COL_064_63"                 
"Z_COL_064","A3"."Z_COL_065_64" "Z_COL_065","A3"."Z_COL_066_65"                 
"Z_COL_066","A3"."Z_COL_067_66" "Z_COL_067","A3"."Z_COL_068_67"                 
"Z_COL_068","A3"."Z_COL_069_68" "Z_COL_069","A3"."Z_COL_070_69"                 
"Z_COL_070","A3"."Z_COL_071_70" "Z_COL_071","A3"."Z_COL_072_71"                 
"Z_COL_072","A3"."Z_COL_073_72" "Z_COL_073","A3"."Z_COL_074_73"                 
"Z_COL_074","A3"."Z_COL_075_74" "Z_COL_075","A3"."Z_COL_076_75"                 
"Z_COL_076","A3"."Z_COL_077_76" "Z_COL_077","A3"."Z_COL_078_77"                 
"Z_COL_078","A3"."Z_COL_079_78" "Z_COL_079","A3"."Z_COL_080_79"                 
"Z_COL_080","A3"."Z_COL_081_80" "Z_COL_081","A3"."Z_COL_082_81"                 
"Z_COL_082","A3"."Z_COL_083_82" "Z_COL_083","A3"."Z_COL_084_83"                 
"Z_COL_084","A3"."Z_COL_085_84" "Z_COL_085","A3"."Z_COL_086_85"                 
"Z_COL_086","A3"."Z_COL_087_86" "Z_COL_087","A3"."Z_COL_088_87"                 
"Z_COL_088","A3"."Z_COL_089_88" "Z_COL_089","A3"."Z_COL_090_89"                 
"Z_COL_090","A3"."Z_COL_091_90" "Z_COL_091","A3"."Z_COL_092_91"                 
"Z_COL_092","A3"."Z_COL_093_92" "Z_COL_093","A3"."Z_COL_094_93"                 
"Z_COL_094","A3"."Z_COL_095_94" "Z_COL_095","A3"."Z_COL_096_95"                 
"Z_COL_096","A3"."Z_COL_097_96" "Z_COL_097","A3"."Z_COL_098_97"                 
"Z_COL_098","A3"."Z_COL_099_98" "Z_COL_099","A3"."Z_COL_100_99"                 
"Z_COL_100","A3"."Z_COL_101_100" "Z_COL_101","A3"."Z_COL_102_101"               
"Z_COL_102","A3"."Z_COL_103_102" "Z_COL_103","A3"."Z_COL_104_103"               
"Z_COL_104","A3"."Z_COL_105_104" "Z_COL_105","A3"."Z_COL_106_105"               
"Z_COL_106","A3"."Z_COL_107_106" "Z_COL_107","A3"."Z_COL_108_107"               
"Z_COL_108","A3"."Z_COL_109_108" "Z_COL_109","A3"."Z_COL_110_109"               
"Z_COL_110","A3"."Z_COL_111_110" "Z_COL_111","A3"."Z_COL_112_111"               
"Z_COL_112","A3"."Z_COL_113_112" "Z_COL_113","A3"."Z_COL_114_113"               
"Z_COL_114","A3"."Z_COL_115_114" "Z_COL_115","A3"."Z_COL_116_115"               
"Z_COL_116","A3"."Z_COL_117_116" "Z_COL_117","A3"."Z_COL_118_117"               
"Z_COL_118","A3"."Z_COL_119_118" "Z_COL_119","A3"."Z_COL_120_119"               
"Z_COL_120","A3"."Z_COL_121_120" "Z_COL_121","A3"."Z_COL_122_121"               
"Z_COL_122","A3"."Z_COL_123_122" "Z_COL_123","A3"."Z_COL_124_123"               
"Z_COL_124","A3"."Z_COL_125_124" "Z_COL_125","A3"."Z_COL_126_125"               
"Z_COL_126","A3"."Z_COL_127_126" "Z_COL_127","A3"."Z_COL_128_127"               
"Z_COL_128","A3"."Z_COL_129_128" "Z_COL_129","A3"."Z_COL_130_129"               
"Z_COL_130","A3"."Z_COL_131_130" "Z_COL_131","A3"."Z_COL_132_131"               
"Z_COL_132","A3"."Z_COL_133_132" "Z_COL_133","A3"."Z_COL_134_133"               
"Z_COL_134","A3"."Z_COL_135_134" "Z_COL_135","A3"."Z_COL_136_135"               
"Z_COL_136","A3"."Z_COL_137_136" "Z_COL_137","A3"."Z_COL_138_137"               
"Z_COL_138","A3"."Z_COL_139_138" "Z_COL_139","A3"."Z_COL_140_139"               
"Z_COL_140","A3"."Z_COL_141_140" "Z_COL_141","A3"."Z_COL_142_141"               
"Z_COL_142","A3"."Z_COL_143_142" "Z_COL_143","A3"."Z_COL_144_143"               
"Z_COL_144","A3"."Z_COL_145_144" "Z_COL_145","A3"."Z_COL_146_145"               
"Z_COL_146","A3"."Z_COL_147_146" "Z_COL_147","A3"."Z_COL_148_147"               
"Z_COL_148","A3"."Z_COL_149_148" "Z_COL_149","A3"."Z_COL_150_149"               
"Z_COL_150","A3"."Z_COL_151_150" "Z_COL_151","A3"."Z_COL_152_151"               
"Z_COL_152","A3"."Z_COL_153_152" "Z_COL_153","A3"."Z_COL_154_153"               
"Z_COL_154","A3"."Z_COL_155_154" "Z_COL_155","A3"."Z_COL_156_155"               
"Z_COL_156","A3"."Z_COL_157_156" "Z_COL_157","A3"."Z_COL_158_157"               
"Z_COL_158","A3"."Z_COL_159_158" "Z_COL_159","A3"."Z_COL_160_159"               
"Z_COL_160","A3"."Z_COL_161_160" "Z_COL_161","A3"."Z_COL_162_161"               
"Z_COL_162","A3"."Z_COL_163_162" "Z_COL_163","A3"."Z_COL_164_163"               
"Z_COL_164","A3"."Z_COL_165_164" "Z_COL_165","A3"."Z_COL_166_165"               
"Z_COL_166","A3"."Z_COL_167_166" "Z_COL_167","A3"."Z_COL_168_167"               
"Z_COL_168","A3"."Z_COL_169_168" "Z_COL_169","A3"."Z_COL_170_169"               
"Z_COL_170","A3"."Z_COL_171_170" "Z_COL_171","A3"."Z_COL_172_171"               
"Z_COL_172","A3"."Z_COL_173_172" "Z_COL_173","A3"."Z_COL_174_173"               
"Z_COL_174","A3"."Z_COL_175_174" "Z_COL_175","A3"."Z_COL_176_175"               
"Z_COL_176","A3"."Z_COL_177_176" "Z_COL_177","A3"."Z_COL_178_177"               
"Z_COL_178","A3"."Z_COL_179_178" "Z_COL_179","A3"."Z_COL_180_179"               
"Z_COL_180","A3"."Z_COL_181_180" "Z_COL_181","A3"."Z_COL_182_181"               
"Z_COL_182","A3"."Z_COL_183_182" "Z_COL_183","A3"."Z_COL_184_183"               
"Z_COL_184","A3"."Z_COL_185_184" "Z_COL_185","A3"."Z_COL_186_185"               
"Z_COL_186","A3"."Z_COL_187_186" "Z_COL_187","A3"."Z_COL_188_187"               
"Z_COL_188","A3"."Z_COL_189_188" "Z_COL_189","A3"."Z_COL_190_189"               
"Z_COL_190","A3"."Z_COL_191_190" "Z_COL_191","A3"."Z_COL_192_191"               
"Z_COL_192","A3"."Z_COL_193_192" "Z_COL_193","A3"."Z_COL_194_193"               
"Z_COL_194","A3"."Z_COL_195_194" "Z_COL_195","A3"."Z_COL_196_195"               
"Z_COL_196","A3"."Z_COL_197_196" "Z_COL_197","A3"."Z_COL_198_197"               
"Z_COL_198","A3"."Z_COL_199_198" "Z_COL_199","A3"."Z_COL_200_199"               
"Z_COL_200","A3"."Z_COL_201_200" "Z_COL_201","A3"."Z_COL_202_201"               
"Z_COL_202","A3"."Z_COL_203_202" "Z_COL_203","A3"."Z_COL_204_203"               
"Z_COL_204","A3"."Z_COL_205_204" "Z_COL_205","A3"."Z_COL_206_205"               
"Z_COL_206","A3"."Z_COL_207_206" "Z_COL_207","A3"."Z_COL_208_207"               
"Z_COL_208","A3"."Z_COL_209_208" "Z_COL_209","A3"."Z_COL_210_209"               
"Z_COL_210","A3"."Z_COL_211_210" "Z_COL_211","A3"."Z_COL_212_211"               
"Z_COL_212","A3"."Z_COL_213_212" "Z_COL_213","A3"."Z_COL_214_213"               
"Z_COL_214","A3"."Z_COL_215_214" "Z_COL_215","A3"."Z_COL_216_215"               
"Z_COL_216","A3"."Z_COL_217_216" "Z_COL_217","A3"."Z_COL_218_217"               
"Z_COL_218","A3"."Z_COL_219_218" "Z_COL_219","A3"."Z_COL_220_219"               
"Z_COL_220","A3"."Z_COL_221_220" "Z_COL_221","A3"."Z_COL_222_221"               
"Z_COL_222","A3"."Z_COL_223_222" "Z_COL_223","A3"."Z_COL_224_223"               
"Z_COL_224","A3"."Z_COL_225_224" "Z_COL_225","A3"."Z_COL_226_225"               
"Z_COL_226","A3"."Z_COL_227_226" "Z_COL_227","A3"."Z_COL_228_227"               
"Z_COL_228","A3"."Z_COL_229_228" "Z_COL_229","A3"."Z_COL_230_229"               
"Z_COL_230","A3"."Z_COL_231_230" "Z_COL_231","A3"."Z_COL_232_231"               
"Z_COL_232","A3"."Z_COL_233_232" "Z_COL_233","A3"."Z_COL_234_233"               
"Z_COL_234","A3"."Z_COL_235_234" "Z_COL_235","A3"."Z_COL_236_235"               
"Z_COL_236","A3"."Z_COL_237_236" "Z_COL_237","A3"."Z_COL_238_237"               
"Z_COL_238","A3"."Z_COL_239_238" "Z_COL_239","A3"."Z_COL_240_239"               
"Z_COL_240","A3"."Z_COL_241_240" "Z_COL_241","A3"."Z_COL_242_241"               
"Z_COL_242","A3"."Z_COL_243_242" "Z_COL_243","A3"."Z_COL_244_243"               
"Z_COL_244","A3"."Z_COL_245_244" "Z_COL_245","A3"."Z_COL_246_245"               
"Z_COL_246","A3"."Z_COL_247_246" "Z_COL_247","A3"."Z_COL_248_247"               
"Z_COL_248","A3"."Z_COL_249_248" "Z_COL_249","A3"."Z_COL_250_249"               
"Z_COL_250","A3"."Z_COL_251_250" "Z_COL_251","A3"."Z_COL_252_251"               
"Z_COL_252","A3"."Z_COL_253_252" "Z_COL_253","A3"."Z_COL_254_253"               
"Z_COL_254","A3"."Z_COL_255_254" "Z_COL_255","A3"."Z_COL_256_255"               
"Z_COL_256","A3"."Z_COL_257_256" "Z_COL_257","A3"."Z_COL_258_257"               
"Z_COL_258","A3"."Z_COL_259_258" "Z_COL_259","A3"."Z_COL_260_259"               
"Z_COL_260","A3"."Z_COL_261_260" "Z_COL_261","A3"."Z_COL_262_261"               
"Z_COL_262","A3"."Z_COL_263_262" "Z_COL_263","A3"."Z_COL_264_263"               
"Z_COL_264","A3"."Z_COL_265_264" "Z_COL_265","A3"."Z_COL_266_265"               
"Z_COL_266","A3"."Z_COL_267_266" "Z_COL_267","A3"."Z_COL_268_267"               
"Z_COL_268","A3"."Z_COL_269_268" "Z_COL_269","A3"."Z_COL_270_269"               
"Z_COL_270","A3"."Z_COL_271_270" "Z_COL_271","A3"."Z_COL_272_271"               
"Z_COL_272","A3"."Z_COL_273_272" "Z_COL_273","A3"."Z_COL_274_273"               
"Z_COL_274","A3"."Z_COL_275_274" "Z_COL_275","A3"."Z_COL_276_275"               
"Z_COL_276","A3"."Z_COL_277_276" "Z_COL_277","A3"."Z_COL_278_277"               
"Z_COL_278","A3"."Z_COL_279_278" "Z_COL_279","A3"."Z_COL_280_279"               
"Z_COL_280","A3"."Z_COL_281_280" "Z_COL_281","A3"."Z_COL_282_281"               
"Z_COL_282","A3"."Z_COL_283_282" "Z_COL_283","A3"."Z_COL_284_283"               
"Z_COL_284","A3"."Z_COL_285_284" "Z_COL_285","A3"."Z_COL_286_285"               
"Z_COL_286","A3"."Z_COL_287_286" "Z_COL_287","A3"."Z_COL_288_287"               
"Z_COL_288","A3"."Z_COL_289_288" "Z_COL_289","A3"."Z_COL_290_289"               
"Z_COL_290","A3"."Z_COL_291_290" "Z_COL_291","A3"."Z_COL_292_291"               
"Z_COL_292","A3"."Z_COL_293_292" "Z_COL_293","A3"."Z_COL_294_293"               
"Z_COL_294","A3"."Z_COL_295_294" "Z_COL_295","A3"."Z_COL_296_295"               
"Z_COL_296","A3"."Z_COL_297_296" "Z_COL_297","A3"."Z_COL_298_297"               
"Z_COL_298","A3"."Z_COL_299_298" "Z_COL_299","A3"."Z_COL_300_299"               
"Z_COL_300","A3"."Z_COL_301_300" "Z_COL_301","A3"."Z_COL_302_301"               
"Z_COL_302","A3"."Z_COL_303_302" "Z_COL_303","A3"."Z_COL_304_303"               
"Z_COL_304","A3"."Z_COL_305_304" "Z_COL_305","A3"."Z_COL_306_305"               
"Z_COL_306","A3"."Z_COL_307_306" "Z_COL_307","A3"."Z_COL_308_307"               
"Z_COL_308","A3"."Z_COL_309_308" "Z_COL_309","A3"."Z_COL_310_309"               
"Z_COL_310","A3"."Z_COL_311_310" "Z_COL_311","A3"."Z_COL_312_311"               
"Z_COL_312","A3"."Z_COL_313_312" "Z_COL_313","A3"."Z_COL_314_313"               
"Z_COL_314","A3"."Z_COL_315_314" "Z_COL_315","A3"."Z_COL_316_315"               
"Z_COL_316","A3"."Z_COL_317_316" "Z_COL_317","A3"."Z_COL_318_317"               
"Z_COL_318","A3"."Z_COL_319_318" "Z_COL_319","A3"."Z_COL_320_319"               
"Z_COL_320","A3"."Z_COL_321_320" "Z_COL_321","A3"."Z_COL_322_321"               
"Z_COL_322","A3"."Z_COL_323_322" "Z_COL_323","A3"."Z_COL_324_323"               
"Z_COL_324","A3"."Z_COL_325_324" "Z_COL_325","A3"."Z_COL_326_325"               
"Z_COL_326","A3"."Z_COL_327_326" "Z_COL_327","A3"."Z_COL_328_327"               
"Z_COL_328","A3"."Z_COL_329_328" "Z_COL_329","A3"."Z_COL_330_329"               
"Z_COL_330","A3"."Z_COL_331_330" "Z_COL_331","A3"."Z_COL_332_331"               
"Z_COL_332","A3"."Z_COL_333_332" "Z_COL_333","A3"."Z_COL_334_333"               
"Z_COL_334","A3"."Z_COL_335_334" "Z_COL_335","A3"."Z_COL_336_335"               
"Z_COL_336","A3"."Z_COL_337_336" "Z_COL_337","A3"."Z_COL_338_337"               
"Z_COL_338","A3"."Z_COL_339_338" "Z_COL_339","A3"."Z_COL_340_339"               
"Z_COL_340","A3"."Z_COL_341_340" "Z_COL_341","A3"."Z_COL_342_341"               
"Z_COL_342","A3"."Z_COL_343_342" "Z_COL_343","A3"."Z_COL_344_343"               
"Z_COL_344","A3"."Z_COL_345_344" "Z_COL_345","A3"."Z_COL_346_345"               
"Z_COL_346","A3"."Z_COL_347_346" "Z_COL_347","A3"."Z_COL_348_347"               
"Z_COL_348","A3"."Z_COL_349_348" "Z_COL_349","A3"."Z_COL_350_349"               
"Z_COL_350","A3"."Z_COL_351_350" "Z_COL_351","A3"."Z_COL_352_351"               
"Z_COL_352","A3"."Z_COL_353_352" "Z_COL_353","A3"."Z_COL_354_353"               
"Z_COL_354","A3"."Z_COL_355_354" "Z_COL_355","A3"."Z_COL_356_355"               
"Z_COL_356","A3"."Z_COL_357_356" "Z_COL_357","A3"."Z_COL_358_357"               
"Z_COL_358","A3"."Z_COL_359_358" "Z_COL_359","A3"."Z_COL_360_359"               
"Z_COL_360","A3"."Z_COL_361_360" "Z_COL_361","A3"."Z_COL_362_361"               
"Z_COL_362","A3"."Z_COL_363_362" "Z_COL_363","A3"."Z_COL_364_363"               
"Z_COL_364","A3"."Z_COL_365_364" "Z_COL_365","A3"."Z_COL_366_365"               
"Z_COL_366","A3"."Z_COL_367_366" "Z_COL_367","A3"."Z_COL_368_367"               
"Z_COL_368","A3"."Z_COL_369_368" "Z_COL_369","A3"."Z_COL_370_369"               
"Z_COL_370","A3"."Z_COL_371_370" "Z_COL_371","A3"."Z_COL_372_371"               
"Z_COL_372","A3"."Z_COL_373_372" "Z_COL_373","A3"."Z_COL_374_373"               
"Z_COL_374","A3"."Z_COL_375_374" "Z_COL_375","A3"."Z_COL_376_375"               
"Z_COL_376","A3"."Z_COL_377_376" "Z_COL_377","A3"."Z_COL_378_377"               
"Z_COL_378","A3"."Z_COL_379_378" "Z_COL_379","A3"."Z_COL_380_379"               
"Z_COL_380","A3"."Z_COL_381_380" "Z_COL_381","A3"."Z_COL_382_381"               
"Z_COL_382","A3"."Z_COL_383_382" "Z_COL_383","A3"."Z_COL_384_383"               
"Z_COL_384","A3"."Z_COL_385_384" "Z_COL_385","A3"."Z_COL_386_385"               
"Z_COL_386","A3"."Z_COL_387_386" "Z_COL_387","A3"."Z_COL_388_387"               
"Z_COL_388","A3"."Z_COL_389_388" "Z_COL_389","A3"."Z_COL_390_389"               
"Z_COL_390","A3"."Z_COL_391_390" "Z_COL_391","A3"."Z_COL_392_391"               
"Z_COL_392","A3"."Z_COL_393_392" "Z_COL_393","A3"."Z_COL_394_393"               
"Z_COL_394","A3"."Z_COL_395_394" "Z_COL_395","A3"."Z_COL_396_395"               
"Z_COL_396","A3"."Z_COL_397_396" "Z_COL_397","A3"."Z_COL_398_397"               
"Z_COL_398","A3"."Z_COL_399_398" "Z_COL_399","A3"."Z_COL_400_399"               
"Z_COL_400","A3"."Z_COL_401_400" "Z_COL_401","A3"."Z_COL_402_401"               
"Z_COL_402","A3"."Z_COL_403_402" "Z_COL_403","A3"."Z_COL_404_403"               
"Z_COL_404","A3"."Z_COL_405_404" "Z_COL_405","A3"."Z_COL_406_405"               
"Z_COL_406","A3"."Z_COL_407_406" "Z_COL_407","A3"."Z_COL_408_407"               
"Z_COL_408","A3"."Z_COL_409_408" "Z_COL_409","A3"."Z_COL_410_409"               
"Z_COL_410","A3"."Z_COL_411_410" "Z_COL_411","A3"."Z_COL_412_411"               
"Z_COL_412","A3"."Z_COL_413_412" "Z_COL_413","A3"."Z_COL_414_413"               
"Z_COL_414","A3"."Z_COL_415_414" "Z_COL_415","A3"."Z_COL_416_415"               
"Z_COL_416","A3"."Z_COL_417_416" "Z_COL_417","A3"."Z_COL_418_417"               
"Z_COL_418","A3"."Z_COL_419_418" "Z_COL_419","A3"."Z_COL_420_419"               
"Z_COL_420","A3"."Z_COL_421_420" "Z_COL_421","A3"."Z_COL_422_421"               
"Z_COL_422","A3"."Z_COL_423_422" "Z_COL_423","A3"."Z_COL_424_423"               
"Z_COL_424","A3"."Z_COL_425_424" "Z_COL_425","A3"."Z_COL_426_425"               
"Z_COL_426","A3"."Z_COL_427_426" "Z_COL_427","A3"."Z_COL_428_427"               
"Z_COL_428","A3"."Z_COL_429_428" "Z_COL_429","A3"."Z_COL_430_429"               
"Z_COL_430","A3"."Z_COL_431_430" "Z_COL_431","A3"."Z_COL_432_431"               
"Z_COL_432","A3"."Z_COL_433_432" "Z_COL_433","A3"."Z_COL_434_433"               
"Z_COL_434","A3"."Z_COL_435_434" "Z_COL_435","A3"."Z_COL_436_435"               
"Z_COL_436","A3"."Z_COL_437_436" "Z_COL_437","A3"."Z_COL_438_437"               
"Z_COL_438","A3"."Z_COL_439_438" "Z_COL_439","A3"."Z_COL_440_439"               
"Z_COL_440","A3"."Z_COL_441_440" "Z_COL_441","A3"."Z_COL_442_441"               
"Z_COL_442","A3"."Z_COL_443_442" "Z_COL_443","A3"."Z_COL_444_443"               
"Z_COL_444","A3"."Z_COL_445_444" "Z_COL_445","A3"."Z_COL_446_445"               
"Z_COL_446","A3"."Z_COL_447_446" "Z_COL_447","A3"."Z_COL_448_447"               
"Z_COL_448","A3"."Z_COL_449_448" "Z_COL_449","A3"."Z_COL_450_449"               
"Z_COL_450","A3"."Z_COL_451_450" "Z_COL_451","A3"."Z_COL_452_451"               
"Z_COL_452","A3"."Z_COL_453_452" "Z_COL_453","A3"."Z_COL_454_453"               
"Z_COL_454","A3"."Z_COL_455_454" "Z_COL_455","A3"."Z_COL_456_455"               
"Z_COL_456","A3"."Z_COL_457_456" "Z_COL_457","A3"."Z_COL_458_457"               
"Z_COL_458","A3"."Z_COL_459_458" "Z_COL_459","A3"."Z_COL_460_459"               
"Z_COL_460","A3"."Z_COL_461_460" "Z_COL_461","A3"."Z_COL_462_461"               
"Z_COL_462","A3"."Z_COL_463_462" "Z_COL_463","A3"."Z_COL_464_463"               
"Z_COL_464","A3"."Z_COL_465_464" "Z_COL_465","A3"."Z_COL_466_465"               
"Z_COL_466","A3"."Z_COL_467_466" "Z_COL_467","A3"."Z_COL_468_467"               
"Z_COL_468","A3"."Z_COL_469_468" "Z_COL_469","A3"."Z_COL_470_469"               
"Z_COL_470","A3"."Z_COL_471_470" "Z_COL_471","A3"."Z_COL_472_471"               
"Z_COL_472","A3"."Z_COL_473_472" "Z_COL_473","A3"."Z_COL_474_473"               
"Z_COL_474","A3"."Z_COL_475_474" "Z_COL_475","A3"."Z_COL_476_475"               
"Z_COL_476","A3"."Z_COL_477_476" "Z_COL_477","A3"."Z_COL_478_477"               
"Z_COL_478","A3"."Z_COL_479_478" "Z_COL_479","A3"."Z_COL_480_479"               
"Z_COL_480","A3"."Z_COL_481_480" "Z_COL_481","A3"."Z_COL_482_481"               
"Z_COL_482","A3"."Z_COL_483_482" "Z_COL_483","A3"."Z_COL_484_483"               
"Z_COL_484","A3"."Z_COL_485_484" "Z_COL_485","A3"."Z_COL_486_485"               
"Z_COL_486","A3"."Z_COL_487_486" "Z_COL_487","A3"."Z_COL_488_487"               
"Z_COL_488","A3"."Z_COL_489_488" "Z_COL_489","A3"."Z_COL_490_489"               
"Z_COL_490","A3"."Z_COL_491_490" "Z_COL_491","A3"."Z_COL_492_491"               
"Z_COL_492","A3"."Z_COL_493_492" "Z_COL_493","A3"."Z_COL_494_493"               
"Z_COL_494","A3"."Z_COL_495_494" "Z_COL_495","A3"."Z_COL_496_495"               
"Z_COL_496","A3"."Z_COL_497_496" "Z_COL_497","A3"."Z_COL_498_497"               
"Z_COL_498","A3"."Z_COL_499_498" "Z_COL_499","A3"."Z_COL_001_499"               
"Z_COL_001","A3"."T_COL__002_500" "T_COL__002","A3"."T_COL__003_501"            
"T_COL__003","A3"."T_COL__004_502" "T_COL__004","A3"."T_COL__005_503"           
"T_COL__005","A3"."T_COL__006_504" "T_COL__006","A3"."T_COL__007_505"           
"T_COL__007","A3"."T_COL__008_506" "T_COL__008","A3"."T_COL__009_507"           
"T_COL__009","A3"."T_COL__010_508" "T_COL__010","A3"."T_COL__011_509"           
"T_COL__011","A3"."T_COL__012_510" "T_COL__012","A3"."T_COL__013_511"           
"T_COL__013","A3"."T_COL__014_512" "T_COL__014","A3"."T_COL__015_513"           
"T_COL__015","A3"."T_COL__016_514" "T_COL__016","A3"."T_COL__017_515"           
"T_COL__017","A3"."T_COL__018_516" "T_COL__018","A3"."T_COL__019_517"           
"T_COL__019","A3"."T_COL__020_518" "T_COL__020","A3"."T_COL__021_519"           
"T_COL__021","A3"."T_COL__022_520" "T_COL__022","A3"."T_COL__023_521"           
"T_COL__023","A3"."T_COL__024_522" "T_COL__024","A3"."T_COL__025_523"           
"T_COL__025","A3"."T_COL__026_524" "T_COL__026","A3"."T_COL__027_525"           
"T_COL__027","A3"."T_COL__028_526" "T_COL__028","A3"."T_COL__029_527"           
"T_COL__029","A3"."T_COL__030_528" "T_COL__030","A3"."T_COL__031_529"           
"T_COL__031","A3"."T_COL__032_530" "T_COL__032","A3"."T_COL__033_531"           
"T_COL__033","A3"."T_COL__034_532" "T_COL__034","A3"."T_COL__035_533"           
"T_COL__035","A3"."T_COL__036_534" "T_COL__036","A3"."T_COL__037_535"           
"T_COL__037","A3"."T_COL__038_536" "T_COL__038","A3"."T_COL__039_537"           
"T_COL__039","A3"."T_COL__040_538" "T_COL__040","A3"."T_COL__041_539"           
"T_COL__041","A3"."T_COL__042_540" "T_COL__042","A3"."T_COL__043_541"           
"T_COL__043","A3"."T_COL__044_542" "T_COL__044","A3"."T_COL__045_543"           
"T_COL__045","A3"."T_COL__046_544" "T_COL__046","A3"."T_COL__047_545"           
"T_COL__047","A3"."T_COL__048_546" "T_COL__048","A3"."T_COL__049_547"           
"T_COL__049","A3"."T_COL__050_548" "T_COL__050","A3"."T_COL__051_549"           
"T_COL__051","A3"."T_COL__052_550" "T_COL__052","A3"."T_COL__053_551"           
"T_COL__053","A3"."T_COL__054_552" "T_COL__054","A3"."T_COL__055_553"           
"T_COL__055","A3"."T_COL__056_554" "T_COL__056","A3"."T_COL__057_555"           
"T_COL__057","A3"."T_COL__058_556" "T_COL__058","A3"."T_COL__059_557"           
"T_COL__059","A3"."T_COL__060_558" "T_COL__060","A3"."T_COL__061_559"           
"T_COL__061","A3"."T_COL__062_560" "T_COL__062","A3"."T_COL__063_561"           
"T_COL__063","A3"."T_COL__064_562" "T_COL__064","A3"."T_COL__065_563"           
"T_COL__065","A3"."T_COL__066_564" "T_COL__066","A3"."T_COL__067_565"           
"T_COL__067","A3"."T_COL__068_566" "T_COL__068","A3"."T_COL__069_567"           
"T_COL__069","A3"."T_COL__070_568" "T_COL__070","A3"."T_COL__071_569"           
"T_COL__071","A3"."T_COL__072_570" "T_COL__072","A3"."T_COL__073_571"           
"T_COL__073","A3"."T_COL__074_572" "T_COL__074","A3"."T_COL__075_573"           
"T_COL__075","A3"."T_COL__076_574" "T_COL__076","A3"."T_COL__077_575"           
"T_COL__077","A3"."T_COL__078_576" "T_COL__078","A3"."T_COL__079_577"           
"T_COL__079","A3"."T_COL__080_578" "T_COL__080","A3"."T_COL__081_579"           
"T_COL__081","A3"."T_COL__082_580" "T_COL__082","A3"."T_COL__083_581"           
"T_COL__083","A3"."T_COL__084_582" "T_COL__084","A3"."T_COL__085_583"           
"T_COL__085","A3"."T_COL__086_584" "T_COL__086","A3"."T_COL__087_585"           
"T_COL__087","A3"."T_COL__088_586" "T_COL__088","A3"."T_COL__089_587"           
"T_COL__089","A3"."T_COL__090_588" "T_COL__090","A3"."T_COL__091_589"           
"T_COL__091","A3"."T_COL__092_590" "T_COL__092","A3"."T_COL__093_591"           
"T_COL__093","A3"."T_COL__094_592" "T_COL__094","A3"."T_COL__095_593"           
"T_COL__095","A3"."T_COL__096_594" "T_COL__096","A3"."T_COL__097_595"           
"T_COL__097","A3"."T_COL__098_596" "T_COL__098","A3"."T_COL__099_597"           
"T_COL__099","A3"."T_COL__100_598" "T_COL__100","A3"."T_COL__101_599"           
"T_COL__101","A3"."T_COL__102_                                                  
                                                                                

The output generated by the procedure is cut short and doesn’t provide the full expansion list, however it should be easily seen that Oracle is doing something ‘interesting’ when expanding the ANSI join. Again, it doesn’t occur just with ‘select *’ queries, as a single-column subquery also produces the same results:


SQL> 
SQL> --********************************************************************************--
SQL> -- sub selecting only one column also fails in Oracle 12c:
SQL> -- SQL Error: ORA-01792: maximum number of columns in a table or view is 1000
SQL> --********************************************************************************--
SQL> 
SQL> select (select 1 from dual where 1 = 2) as exp_column
  2  from ORANGE g
  3    join MAUVE r on g.Z_COL_001 = r.Z_COL_001
  4    join YELLOW p on g.Z_COL_001 = p.Z_COL_001;

no rows selected

SQL> 
SQL> --********************************************************************************--
SQL> -- The same occurs here
SQL> -- Because of the errant expansion the total number of columns expands to just under
SQL> -- the query limit so that even selecting 1 column from causes the error to occur
SQL> --********************************************************************************--
SQL> set serveroutput on
SQL> set long 20000
SQL> 
SQL> variable m_sql_out clob
SQL> 
SQL> declare
  2      m_sql_in    clob :=
  3                          '
  4                          select (select 1 from dual where 1 = 2) as exp_column
  5                          from ORANGE g
  6                          join MAUVE r on g.Z_COL_001 = r.Z_COL_001
  7                          join YELLOW p on g.Z_COL_001 = p.Z_COL_001
  8                          ';
  9  begin
 10  
 11      dbms_utility.expand_sql_text(
 12          m_sql_in,
 13          :m_sql_out
 14      );
 15  
 16  end;
 17  /

PL/SQL procedure successfully completed.

SQL> 
SQL> column m_sql_out wrap word
SQL> print m_sql_out
SELECT  (SELECT 1 "1" FROM "SYS"."DUAL" "A2" WHERE 1=2) "EXP_COLUMN" FROM       
(SELECT "A4"."Z_COL_001_0" "Z_COL_001","A4"."Z_COL_002_1"                       
"Z_COL_002","A4"."Z_COL_003_2" "Z_COL_003","A4"."Z_COL_004_3"                   
"Z_COL_004","A4"."Z_COL_005_4" "Z_COL_005","A4"."Z_COL_006_5"                   
"Z_COL_006","A4"."Z_COL_007_6" "Z_COL_007","A4"."Z_COL_008_7"                   
"Z_COL_008","A4"."Z_COL_009_8" "Z_COL_009","A4"."Z_COL_010_9"                   
"Z_COL_010","A4"."Z_COL_011_10" "Z_COL_011","A4"."Z_COL_012_11"                 
"Z_COL_012","A4"."Z_COL_013_12" "Z_COL_013","A4"."Z_COL_014_13"                 
"Z_COL_014","A4"."Z_COL_015_14" "Z_COL_015","A4"."Z_COL_016_15"                 
"Z_COL_016","A4"."Z_COL_017_16" "Z_COL_017","A4"."Z_COL_018_17"                 
"Z_COL_018","A4"."Z_COL_019_18" "Z_COL_019","A4"."Z_COL_020_19"                 
"Z_COL_020","A4"."Z_COL_021_20" "Z_COL_021","A4"."Z_COL_022_21"                 
"Z_COL_022","A4"."Z_COL_023_22" "Z_COL_023","A4"."Z_COL_024_23"                 
"Z_COL_024","A4"."Z_COL_025_24" "Z_COL_025","A4"."Z_COL_026_25"                 
"Z_COL_026","A4"."Z_COL_027_26" "Z_COL_027","A4"."Z_COL_028_27"                 
"Z_COL_028","A4"."Z_COL_029_28" "Z_COL_029","A4"."Z_COL_030_29"                 
"Z_COL_030","A4"."Z_COL_031_30" "Z_COL_031","A4"."Z_COL_032_31"                 
"Z_COL_032","A4"."Z_COL_033_32" "Z_COL_033","A4"."Z_COL_034_33"                 
"Z_COL_034","A4"."Z_COL_035_34" "Z_COL_035","A4"."Z_COL_036_35"                 
"Z_COL_036","A4"."Z_COL_037_36" "Z_COL_037","A4"."Z_COL_038_37"                 
"Z_COL_038","A4"."Z_COL_039_38" "Z_COL_039","A4"."Z_COL_040_39"                 
"Z_COL_040","A4"."Z_COL_041_40" "Z_COL_041","A4"."Z_COL_042_41"                 
"Z_COL_042","A4"."Z_COL_043_42" "Z_COL_043","A4"."Z_COL_044_43"                 
"Z_COL_044","A4"."Z_COL_045_44" "Z_COL_045","A4"."Z_COL_046_45"                 
"Z_COL_046","A4"."Z_COL_047_46" "Z_COL_047","A4"."Z_COL_048_47"                 
"Z_COL_048","A4"."Z_COL_049_48" "Z_COL_049","A4"."Z_COL_050_49"                 
"Z_COL_050","A4"."Z_COL_051_50" "Z_COL_051","A4"."Z_COL_052_51"                 
"Z_COL_052","A4"."Z_COL_053_52" "Z_COL_053","A4"."Z_COL_054_53"                 
"Z_COL_054","A4"."Z_COL_055_54" "Z_COL_055","A4"."Z_COL_056_55"                 
"Z_COL_056","A4"."Z_COL_057_56" "Z_COL_057","A4"."Z_COL_058_57"                 
"Z_COL_058","A4"."Z_COL_059_58" "Z_COL_059","A4"."Z_COL_060_59"                 
"Z_COL_060","A4"."Z_COL_061_60" "Z_COL_061","A4"."Z_COL_062_61"                 
"Z_COL_062","A4"."Z_COL_063_62" "Z_COL_063","A4"."Z_COL_064_63"                 
"Z_COL_064","A4"."Z_COL_065_64" "Z_COL_065","A4"."Z_COL_066_65"                 
"Z_COL_066","A4"."Z_COL_067_66" "Z_COL_067","A4"."Z_COL_068_67"                 
"Z_COL_068","A4"."Z_COL_069_68" "Z_COL_069","A4"."Z_COL_070_69"                 
"Z_COL_070","A4"."Z_COL_071_70" "Z_COL_071","A4"."Z_COL_072_71"                 
"Z_COL_072","A4"."Z_COL_073_72" "Z_COL_073","A4"."Z_COL_074_73"                 
"Z_COL_074","A4"."Z_COL_075_74" "Z_COL_075","A4"."Z_COL_076_75"                 
"Z_COL_076","A4"."Z_COL_077_76" "Z_COL_077","A4"."Z_COL_078_77"                 
"Z_COL_078","A4"."Z_COL_079_78" "Z_COL_079","A4"."Z_COL_080_79"                 
"Z_COL_080","A4"."Z_COL_081_80" "Z_COL_081","A4"."Z_COL_082_81"                 
"Z_COL_082","A4"."Z_COL_083_82" "Z_COL_083","A4"."Z_COL_084_83"                 
"Z_COL_084","A4"."Z_COL_085_84" "Z_COL_085","A4"."Z_COL_086_85"                 
"Z_COL_086","A4"."Z_COL_087_86" "Z_COL_087","A4"."Z_COL_088_87"                 
"Z_COL_088","A4"."Z_COL_089_88" "Z_COL_089","A4"."Z_COL_090_89"                 
"Z_COL_090","A4"."Z_COL_091_90" "Z_COL_091","A4"."Z_COL_092_91"                 
"Z_COL_092","A4"."Z_COL_093_92" "Z_COL_093","A4"."Z_COL_094_93"                 
"Z_COL_094","A4"."Z_COL_095_94" "Z_COL_095","A4"."Z_COL_096_95"                 
"Z_COL_096","A4"."Z_COL_097_96" "Z_COL_097","A4"."Z_COL_098_97"                 
"Z_COL_098","A4"."Z_COL_099_98" "Z_COL_099","A4"."Z_COL_100_99"                 
"Z_COL_100","A4"."Z_COL_101_100" "Z_COL_101","A4"."Z_COL_102_101"               
"Z_COL_102","A4"."Z_COL_103_102" "Z_COL_103","A4"."Z_COL_104_103"               
"Z_COL_104","A4"."Z_COL_105_104" "Z_COL_105","A4"."Z_COL_106_105"               
"Z_COL_106","A4"."Z_COL_107_106" "Z_COL_107","A4"."Z_COL_108_107"               
"Z_COL_108","A4"."Z_COL_109_108" "Z_COL_109","A4"."Z_COL_110_109"               
"Z_COL_110","A4"."Z_COL_111_110" "Z_COL_111","A4"."Z_COL_112_111"               
"Z_COL_112","A4"."Z_COL_113_112" "Z_COL_113","A4"."Z_COL_114_113"               
"Z_COL_114","A4"."Z_COL_115_114" "Z_COL_115","A4"."Z_COL_116_115"               
"Z_COL_116","A4"."Z_COL_117_116" "Z_COL_117","A4"."Z_COL_118_117"               
"Z_COL_118","A4"."Z_COL_119_118" "Z_COL_119","A4"."Z_COL_120_119"               
"Z_COL_120","A4"."Z_COL_121_120" "Z_COL_121","A4"."Z_COL_122_121"               
"Z_COL_122","A4"."Z_COL_123_122" "Z_COL_123","A4"."Z_COL_124_123"               
"Z_COL_124","A4"."Z_COL_125_124" "Z_COL_125","A4"."Z_COL_126_125"               
"Z_COL_126","A4"."Z_COL_127_126" "Z_COL_127","A4"."Z_COL_128_127"               
"Z_COL_128","A4"."Z_COL_129_128" "Z_COL_129","A4"."Z_COL_130_129"               
"Z_COL_130","A4"."Z_COL_131_130" "Z_COL_131","A4"."Z_COL_132_131"               
"Z_COL_132","A4"."Z_COL_133_132" "Z_COL_133","A4"."Z_COL_134_133"               
"Z_COL_134","A4"."Z_COL_135_134" "Z_COL_135","A4"."Z_COL_136_135"               
"Z_COL_136","A4"."Z_COL_137_136" "Z_COL_137","A4"."Z_COL_138_137"               
"Z_COL_138","A4"."Z_COL_139_138" "Z_COL_139","A4"."Z_COL_140_139"               
"Z_COL_140","A4"."Z_COL_141_140" "Z_COL_141","A4"."Z_COL_142_141"               
"Z_COL_142","A4"."Z_COL_143_142" "Z_COL_143","A4"."Z_COL_144_143"               
"Z_COL_144","A4"."Z_COL_145_144" "Z_COL_145","A4"."Z_COL_146_145"               
"Z_COL_146","A4"."Z_COL_147_146" "Z_COL_147","A4"."Z_COL_148_147"               
"Z_COL_148","A4"."Z_COL_149_148" "Z_COL_149","A4"."Z_COL_150_149"               
"Z_COL_150","A4"."Z_COL_151_150" "Z_COL_151","A4"."Z_COL_152_151"               
"Z_COL_152","A4"."Z_COL_153_152" "Z_COL_153","A4"."Z_COL_154_153"               
"Z_COL_154","A4"."Z_COL_155_154" "Z_COL_155","A4"."Z_COL_156_155"               
"Z_COL_156","A4"."Z_COL_157_156" "Z_COL_157","A4"."Z_COL_158_157"               
"Z_COL_158","A4"."Z_COL_159_158" "Z_COL_159","A4"."Z_COL_160_159"               
"Z_COL_160","A4"."Z_COL_161_160" "Z_COL_161","A4"."Z_COL_162_161"               
"Z_COL_162","A4"."Z_COL_163_162" "Z_COL_163","A4"."Z_COL_164_163"               
"Z_COL_164","A4"."Z_COL_165_164" "Z_COL_165","A4"."Z_COL_166_165"               
"Z_COL_166","A4"."Z_COL_167_166" "Z_COL_167","A4"."Z_COL_168_167"               
"Z_COL_168","A4"."Z_COL_169_168" "Z_COL_169","A4"."Z_COL_170_169"               
"Z_COL_170","A4"."Z_COL_171_170" "Z_COL_171","A4"."Z_COL_172_171"               
"Z_COL_172","A4"."Z_COL_173_172" "Z_COL_173","A4"."Z_COL_174_173"               
"Z_COL_174","A4"."Z_COL_175_174" "Z_COL_175","A4"."Z_COL_176_175"               
"Z_COL_176","A4"."Z_COL_177_176" "Z_COL_177","A4"."Z_COL_178_177"               
"Z_COL_178","A4"."Z_COL_179_178" "Z_COL_179","A4"."Z_COL_180_179"               
"Z_COL_180","A4"."Z_COL_181_180" "Z_COL_181","A4"."Z_COL_182_181"               
"Z_COL_182","A4"."Z_COL_183_182" "Z_COL_183","A4"."Z_COL_184_183"               
"Z_COL_184","A4"."Z_COL_185_184" "Z_COL_185","A4"."Z_COL_186_185"               
"Z_COL_186","A4"."Z_COL_187_186" "Z_COL_187","A4"."Z_COL_188_187"               
"Z_COL_188","A4"."Z_COL_189_188" "Z_COL_189","A4"."Z_COL_190_189"               
"Z_COL_190","A4"."Z_COL_191_190" "Z_COL_191","A4"."Z_COL_192_191"               
"Z_COL_192","A4"."Z_COL_193_192" "Z_COL_193","A4"."Z_COL_194_193"               
"Z_COL_194","A4"."Z_COL_195_194" "Z_COL_195","A4"."Z_COL_196_195"               
"Z_COL_196","A4"."Z_COL_197_196" "Z_COL_197","A4"."Z_COL_198_197"               
"Z_COL_198","A4"."Z_COL_199_198" "Z_COL_199","A4"."Z_COL_200_199"               
"Z_COL_200","A4"."Z_COL_201_200" "Z_COL_201","A4"."Z_COL_202_201"               
"Z_COL_202","A4"."Z_COL_203_202" "Z_COL_203","A4"."Z_COL_204_203"               
"Z_COL_204","A4"."Z_COL_205_204" "Z_COL_205","A4"."Z_COL_206_205"               
"Z_COL_206","A4"."Z_COL_207_206" "Z_COL_207","A4"."Z_COL_208_207"               
"Z_COL_208","A4"."Z_COL_209_208" "Z_COL_209","A4"."Z_COL_210_209"               
"Z_COL_210","A4"."Z_COL_211_210" "Z_COL_211","A4"."Z_COL_212_211"               
"Z_COL_212","A4"."Z_COL_213_212" "Z_COL_213","A4"."Z_COL_214_213"               
"Z_COL_214","A4"."Z_COL_215_214" "Z_COL_215","A4"."Z_COL_216_215"               
"Z_COL_216","A4"."Z_COL_217_216" "Z_COL_217","A4"."Z_COL_218_217"               
"Z_COL_218","A4"."Z_COL_219_218" "Z_COL_219","A4"."Z_COL_220_219"               
"Z_COL_220","A4"."Z_COL_221_220" "Z_COL_221","A4"."Z_COL_222_221"               
"Z_COL_222","A4"."Z_COL_223_222" "Z_COL_223","A4"."Z_COL_224_223"               
"Z_COL_224","A4"."Z_COL_225_224" "Z_COL_225","A4"."Z_COL_226_225"               
"Z_COL_226","A4"."Z_COL_227_226" "Z_COL_227","A4"."Z_COL_228_227"               
"Z_COL_228","A4"."Z_COL_229_228" "Z_COL_229","A4"."Z_COL_230_229"               
"Z_COL_230","A4"."Z_COL_231_230" "Z_COL_231","A4"."Z_COL_232_231"               
"Z_COL_232","A4"."Z_COL_233_232" "Z_COL_233","A4"."Z_COL_234_233"               
"Z_COL_234","A4"."Z_COL_235_234" "Z_COL_235","A4"."Z_COL_236_235"               
"Z_COL_236","A4"."Z_COL_237_236" "Z_COL_237","A4"."Z_COL_238_237"               
"Z_COL_238","A4"."Z_COL_239_238" "Z_COL_239","A4"."Z_COL_240_239"               
"Z_COL_240","A4"."Z_COL_241_240" "Z_COL_241","A4"."Z_COL_242_241"               
"Z_COL_242","A4"."Z_COL_243_242" "Z_COL_243","A4"."Z_COL_244_243"               
"Z_COL_244","A4"."Z_COL_245_244" "Z_COL_245","A4"."Z_COL_246_245"               
"Z_COL_246","A4"."Z_COL_247_246" "Z_COL_247","A4"."Z_COL_248_247"               
"Z_COL_248","A4"."Z_COL_249_248" "Z_COL_249","A4"."Z_COL_250_249"               
"Z_COL_250","A4"."Z_COL_251_250" "Z_COL_251","A4"."Z_COL_252_251"               
"Z_COL_252","A4"."Z_COL_253_252" "Z_COL_253","A4"."Z_COL_254_253"               
"Z_COL_254","A4"."Z_COL_255_254" "Z_COL_255","A4"."Z_COL_256_255"               
"Z_COL_256","A4"."Z_COL_257_256" "Z_COL_257","A4"."Z_COL_258_257"               
"Z_COL_258","A4"."Z_COL_259_258" "Z_COL_259","A4"."Z_COL_260_259"               
"Z_COL_260","A4"."Z_COL_261_260" "Z_COL_261","A4"."Z_COL_262_261"               
"Z_COL_262","A4"."Z_COL_263_262" "Z_COL_263","A4"."Z_COL_264_263"               
"Z_COL_264","A4"."Z_COL_265_264" "Z_COL_265","A4"."Z_COL_266_265"               
"Z_COL_266","A4"."Z_COL_267_266" "Z_COL_267","A4"."Z_COL_268_267"               
"Z_COL_268","A4"."Z_COL_269_268" "Z_COL_269","A4"."Z_COL_270_269"               
"Z_COL_270","A4"."Z_COL_271_270" "Z_COL_271","A4"."Z_COL_272_271"               
"Z_COL_272","A4"."Z_COL_273_272" "Z_COL_273","A4"."Z_COL_274_273"               
"Z_COL_274","A4"."Z_COL_275_274" "Z_COL_275","A4"."Z_COL_276_275"               
"Z_COL_276","A4"."Z_COL_277_276" "Z_COL_277","A4"."Z_COL_278_277"               
"Z_COL_278","A4"."Z_COL_279_278" "Z_COL_279","A4"."Z_COL_280_279"               
"Z_COL_280","A4"."Z_COL_281_280" "Z_COL_281","A4"."Z_COL_282_281"               
"Z_COL_282","A4"."Z_COL_283_282" "Z_COL_283","A4"."Z_COL_284_283"               
"Z_COL_284","A4"."Z_COL_285_284" "Z_COL_285","A4"."Z_COL_286_285"               
"Z_COL_286","A4"."Z_COL_287_286" "Z_COL_287","A4"."Z_COL_288_287"               
"Z_COL_288","A4"."Z_COL_289_288" "Z_COL_289","A4"."Z_COL_290_289"               
"Z_COL_290","A4"."Z_COL_291_290" "Z_COL_291","A4"."Z_COL_292_291"               
"Z_COL_292","A4"."Z_COL_293_292" "Z_COL_293","A4"."Z_COL_294_293"               
"Z_COL_294","A4"."Z_COL_295_294" "Z_COL_295","A4"."Z_COL_296_295"               
"Z_COL_296","A4"."Z_COL_297_296" "Z_COL_297","A4"."Z_COL_298_297"               
"Z_COL_298","A4"."Z_COL_299_298" "Z_COL_299","A4"."Z_COL_300_299"               
"Z_COL_300","A4"."Z_COL_301_300" "Z_COL_301","A4"."Z_COL_302_301"               
"Z_COL_302","A4"."Z_COL_303_302" "Z_COL_303","A4"."Z_COL_304_303"               
"Z_COL_304","A4"."Z_COL_305_304" "Z_COL_305","A4"."Z_COL_306_305"               
"Z_COL_306","A4"."Z_COL_307_306" "Z_COL_307","A4"."Z_COL_308_307"               
"Z_COL_308","A4"."Z_COL_309_308" "Z_COL_309","A4"."Z_COL_310_309"               
"Z_COL_310","A4"."Z_COL_311_310" "Z_COL_311","A4"."Z_COL_312_311"               
"Z_COL_312","A4"."Z_COL_313_312" "Z_COL_313","A4"."Z_COL_314_313"               
"Z_COL_314","A4"."Z_COL_315_314" "Z_COL_315","A4"."Z_COL_316_315"               
"Z_COL_316","A4"."Z_COL_317_316" "Z_COL_317","A4"."Z_COL_318_317"               
"Z_COL_318","A4"."Z_COL_319_318" "Z_COL_319","A4"."Z_COL_320_319"               
"Z_COL_320","A4"."Z_COL_321_320" "Z_COL_321","A4"."Z_COL_322_321"               
"Z_COL_322","A4"."Z_COL_323_322" "Z_COL_323","A4"."Z_COL_324_323"               
"Z_COL_324","A4"."Z_COL_325_324" "Z_COL_325","A4"."Z_COL_326_325"               
"Z_COL_326","A4"."Z_COL_327_326" "Z_COL_327","A4"."Z_COL_328_327"               
"Z_COL_328","A4"."Z_COL_329_328" "Z_COL_329","A4"."Z_COL_330_329"               
"Z_COL_330","A4"."Z_COL_331_330" "Z_COL_331","A4"."Z_COL_332_331"               
"Z_COL_332","A4"."Z_COL_333_332" "Z_COL_333","A4"."Z_COL_334_333"               
"Z_COL_334","A4"."Z_COL_335_334" "Z_COL_335","A4"."Z_COL_336_335"               
"Z_COL_336","A4"."Z_COL_337_336" "Z_COL_337","A4"."Z_COL_338_337"               
"Z_COL_338","A4"."Z_COL_339_338" "Z_COL_339","A4"."Z_COL_340_339"               
"Z_COL_340","A4"."Z_COL_341_340" "Z_COL_341","A4"."Z_COL_342_341"               
"Z_COL_342","A4"."Z_COL_343_342" "Z_COL_343","A4"."Z_COL_344_343"               
"Z_COL_344","A4"."Z_COL_345_344" "Z_COL_345","A4"."Z_COL_346_345"               
"Z_COL_346","A4"."Z_COL_347_346" "Z_COL_347","A4"."Z_COL_348_347"               
"Z_COL_348","A4"."Z_COL_349_348" "Z_COL_349","A4"."Z_COL_350_349"               
"Z_COL_350","A4"."Z_COL_351_350" "Z_COL_351","A4"."Z_COL_352_351"               
"Z_COL_352","A4"."Z_COL_353_352" "Z_COL_353","A4"."Z_COL_354_353"               
"Z_COL_354","A4"."Z_COL_355_354" "Z_COL_355","A4"."Z_COL_356_355"               
"Z_COL_356","A4"."Z_COL_357_356" "Z_COL_357","A4"."Z_COL_358_357"               
"Z_COL_358","A4"."Z_COL_359_358" "Z_COL_359","A4"."Z_COL_360_359"               
"Z_COL_360","A4"."Z_COL_361_360" "Z_COL_361","A4"."Z_COL_362_361"               
"Z_COL_362","A4"."Z_COL_363_362" "Z_COL_363","A4"."Z_COL_364_363"               
"Z_COL_364","A4"."Z_COL_365_364" "Z_COL_365","A4"."Z_COL_366_365"               
"Z_COL_366","A4"."Z_COL_367_366" "Z_COL_367","A4"."Z_COL_368_367"               
"Z_COL_368","A4"."Z_COL_369_368" "Z_COL_369","A4"."Z_COL_370_369"               
"Z_COL_370","A4"."Z_COL_371_370" "Z_COL_371","A4"."Z_COL_372_371"               
"Z_COL_372","A4"."Z_COL_373_372" "Z_COL_373","A4"."Z_COL_374_373"               
"Z_COL_374","A4"."Z_COL_375_374" "Z_COL_375","A4"."Z_COL_376_375"               
"Z_COL_376","A4"."Z_COL_377_376" "Z_COL_377","A4"."Z_COL_378_377"               
"Z_COL_378","A4"."Z_COL_379_378" "Z_COL_379","A4"."Z_COL_380_379"               
"Z_COL_380","A4"."Z_COL_381_380" "Z_COL_381","A4"."Z_COL_382_381"               
"Z_COL_382","A4"."Z_COL_383_382" "Z_COL_383","A4"."Z_COL_384_383"               
"Z_COL_384","A4"."Z_COL_385_384" "Z_COL_385","A4"."Z_COL_386_385"               
"Z_COL_386","A4"."Z_COL_387_386" "Z_COL_387","A4"."Z_COL_388_387"               
"Z_COL_388","A4"."Z_COL_389_388" "Z_COL_389","A4"."Z_COL_390_389"               
"Z_COL_390","A4"."Z_COL_391_390" "Z_COL_391","A4"."Z_COL_392_391"               
"Z_COL_392","A4"."Z_COL_393_392" "Z_COL_393","A4"."Z_COL_394_393"               
"Z_COL_394","A4"."Z_COL_395_394" "Z_COL_395","A4"."Z_COL_396_395"               
"Z_COL_396","A4"."Z_COL_397_396" "Z_COL_397","A4"."Z_COL_398_397"               
"Z_COL_398","A4"."Z_COL_399_398" "Z_COL_399","A4"."Z_COL_400_399"               
"Z_COL_400","A4"."Z_COL_401_400" "Z_COL_401","A4"."Z_COL_402_401"               
"Z_COL_402","A4"."Z_COL_403_402" "Z_COL_403","A4"."Z_COL_404_403"               
"Z_COL_404","A4"."Z_COL_405_404" "Z_COL_405","A4"."Z_COL_406_405"               
"Z_COL_406","A4"."Z_COL_407_406" "Z_COL_407","A4"."Z_COL_408_407"               
"Z_COL_408","A4"."Z_COL_409_408" "Z_COL_409","A4"."Z_COL_410_409"               
"Z_COL_410","A4"."Z_COL_411_410" "Z_COL_411","A4"."Z_COL_412_411"               
"Z_COL_412","A4"."Z_COL_413_412" "Z_COL_413","A4"."Z_COL_414_413"               
"Z_COL_414","A4"."Z_COL_415_414" "Z_COL_415","A4"."Z_COL_416_415"               
"Z_COL_416","A4"."Z_COL_417_416" "Z_COL_417","A4"."Z_COL_418_417"               
"Z_COL_418","A4"."Z_COL_419_418" "Z_COL_419","A4"."Z_COL_420_419"               
"Z_COL_420","A4"."Z_COL_421_420" "Z_COL_421","A4"."Z_COL_422_421"               
"Z_COL_422","A4"."Z_COL_423_422" "Z_COL_423","A4"."Z_COL_424_423"               
"Z_COL_424","A4"."Z_COL_425_424" "Z_COL_425","A4"."Z_COL_426_425"               
"Z_COL_426","A4"."Z_COL_427_426" "Z_COL_427","A4"."Z_COL_428_427"               
"Z_COL_428","A4"."Z_COL_429_428" "Z_COL_429","A4"."Z_COL_430_429"               
"Z_COL_430","A4"."Z_COL_431_430" "Z_COL_431","A4"."Z_COL_432_431"               
"Z_COL_432","A4"."Z_COL_433_432" "Z_COL_433","A4"."Z_COL_434_433"               
"Z_COL_434","A4"."Z_COL_435_434" "Z_COL_435","A4"."Z_COL_436_435"               
"Z_COL_436","A4"."Z_COL_437_436" "Z_COL_437","A4"."Z_COL_438_437"               
"Z_COL_438","A4"."Z_COL_439_438" "Z_COL_439","A4"."Z_COL_440_439"               
"Z_COL_440","A4"."Z_COL_441_440" "Z_COL_441","A4"."Z_COL_442_441"               
"Z_COL_442","A4"."Z_COL_443_442" "Z_COL_443","A4"."Z_COL_444_443"               
"Z_COL_444","A4"."Z_COL_445_444" "Z_COL_445","A4"."Z_COL_446_445"               
"Z_COL_446","A4"."Z_COL_447_446" "Z_COL_447","A4"."Z_COL_448_447"               
"Z_COL_448","A4"."Z_COL_449_448" "Z_COL_449","A4"."Z_COL_450_449"               
"Z_COL_450","A4"."Z_COL_451_450" "Z_COL_451","A4"."Z_COL_452_451"               
"Z_COL_452","A4"."Z_COL_453_452" "Z_COL_453","A4"."Z_COL_454_453"               
"Z_COL_454","A4"."Z_COL_455_454" "Z_COL_455","A4"."Z_COL_456_455"               
"Z_COL_456","A4"."Z_COL_457_456" "Z_COL_457","A4"."Z_COL_458_457"               
"Z_COL_458","A4"."Z_COL_459_458" "Z_COL_459","A4"."Z_COL_460_459"               
"Z_COL_460","A4"."Z_COL_461_460" "Z_COL_461","A4"."Z_COL_462_461"               
"Z_COL_462","A4"."Z_COL_463_462" "Z_COL_463","A4"."Z_COL_464_463"               
"Z_COL_464","A4"."Z_COL_465_464" "Z_COL_465","A4"."Z_COL_466_465"               
"Z_COL_466","A4"."Z_COL_467_466" "Z_COL_467","A4"."Z_COL_468_467"               
"Z_COL_468","A4"."Z_COL_469_468" "Z_COL_469","A4"."Z_COL_470_469"               
"Z_COL_470","A4"."Z_COL_471_470" "Z_COL_471","A4"."Z_COL_472_471"               
"Z_COL_472","A4"."Z_COL_473_472" "Z_COL_473","A4"."Z_COL_474_473"               
"Z_COL_474","A4"."Z_COL_475_474" "Z_COL_475","A4"."Z_COL_476_475"               
"Z_COL_476","A4"."Z_COL_477_476" "Z_COL_477","A4"."Z_COL_478_477"               
"Z_COL_478","A4"."Z_COL_479_478" "Z_COL_479","A4"."Z_COL_480_479"               
"Z_COL_480","A4"."Z_COL_481_480" "Z_COL_481","A4"."Z_COL_482_481"               
"Z_COL_482","A4"."Z_COL_483_482" "Z_COL_483","A4"."Z_COL_484_483"               
"Z_COL_484","A4"."Z_COL_485_484" "Z_COL_485","A4"."Z_COL_486_485"               
"Z_COL_486","A4"."Z_COL_487_486" "Z_COL_487","A4"."Z_COL_488_487"               
"Z_COL_488","A4"."Z_COL_489_488" "Z_COL_489","A4"."Z_COL_490_489"               
"Z_COL_490","A4"."Z_COL_491_490" "Z_COL_491","A4"."Z_COL_492_491"               
"Z_COL_492","A4"."Z_COL_493_492" "Z_COL_493","A4"."Z_COL_494_493"               
"Z_COL_494","A4"."Z_COL_495_494" "Z_COL_495","A4"."Z_COL_496_495"               
"Z_COL_496","A4"."Z_COL_497_496" "Z_COL_497","A4"."Z_COL_498_497"               
"Z_COL_498","A4"."Z_COL_499_498" "Z_COL_499","A4"."Z_COL_001_499"               
"Z_COL_001","A4"."T_COL__002_500" "T_COL__002","A4"."T_COL__003_501"            
"T_COL__003","A4"."T_COL__004_502" "T_COL__004","A4"."T_COL__005_503"           
"T_COL__005","A4"."T_COL__006_504" "T_COL__006","A4"."T_COL__007_505"           
"T_COL__007","A4"."T_COL__008_506" "T_COL__008","A4"."T_COL__009_507"           
"T_COL__009","A4"."T_COL__010_508" "T_COL__010","A4"."T_COL__011_509"           
"T_COL__011","A4"."T_COL__012_510" "T_COL__012","A4"."T_COL__013_511"           
"T_COL__013","A4"."T_COL__014_512" "T_COL__014","A4"."T_COL__015_513"           
"T_COL__015","A4"."T_COL__016_514" "T_COL__016","A4"."T_COL__017_515"           
"T_COL__017","A4"."T_COL__018_516" "T_COL__018","A4"."T_COL__019_517"           
"T_COL__019","A4"."T_COL__020_518" "T_COL__020","A4"."T_COL__021_519"           
"T_COL__021","A4"."T_COL__022_520" "T_COL__022","A4"."T_COL__023_521"           
"T_COL__023","A4"."T_COL__024_522" "T_COL__024","A4"."T_COL__025_523"           
"T_COL__025","A4"."T_COL__026_524" "T_COL__026","A4"."T_COL__027_525"           
"T_COL__027","A4"."T_COL__028_526" "T_COL__028","A4"."T_COL__029_527"           
"T_COL__029","A4"."T_COL__030_528" "T_COL__030","A4"."T_COL__031_529"           
"T_COL__031","A4"."T_COL__032_530" "T_COL__032","A4"."T_COL__033_531"           
"T_COL__033","A4"."T_COL__034_532" "T_COL__034","A4"."T_COL__035_533"           
"T_COL__035","A4"."T_COL__036_534" "T_COL__036","A4"."T_COL__037_535"           
"T_COL__037","A4"."T_COL__038_536" "T_COL__038","A4"."T_COL__039_537"           
"T_COL__039","A4"."T_COL__040_538" "T_COL__040","A4"."T_COL__041_539"           
"T_COL__041","A4"."T_COL__042_540" "T_COL__042","A4"."T_COL__043_541"           
"T_COL__043","A4"."T_COL__044_542" "T_COL__044","A4"."T_COL__045_543"           
"T_COL__045","A4"."T_COL__046_544" "T_COL__046","A4"."T_COL__047_545"           
"T_COL__047","A4"."T_COL__048_546" "T_COL__048","A4"."T_COL__049_547"           
"T_COL__049","A4"."T_COL__050_548" "T_COL__050","A4"."T_COL__051_549"           
"T_COL__051","A4"."T_COL__052_550" "T_COL__052","A4"."T_COL__053_551"           
"T_COL__053","A4"."T_COL__054_552" "T_COL__054","A4"."T_COL__055_553"           
"T_COL__055","A4"."T_COL__056_554" "T_COL__056","A4"."T_COL__057_555"           
"T_COL__057","A4"."T_COL__058_556" "T_COL__058","A4"."T_COL__059_557"           
"T_COL__059","A4"."T_COL__060_558" "T_COL__060","A4"."T_COL__061_559"           
"T_COL__061","A4"."T_COL__062_560" "T_COL__062","A4"."T_COL__063_561"           
"T_COL__063","A4"."T_COL__064_562" "T_COL__064","A4"."T_COL__065_563"           
"T_COL__065","A4"."T_COL__066_564" "T_COL__066","A4"."T_COL__067_565"           
"T_COL__067","A4"."T_COL__068_566" "T_COL__068","A4"."T_COL__069_567"           
"T_COL__069","A4"."T_COL__070_568" "T_COL__070","A4"."T_COL__071_569"           
"T_COL__071","A4"."T_COL__072_570" "T_COL__072","A4"."T_COL__073_571"           
"T_COL__073","A4"."T_COL__074_572" "T_COL__074","A4"."T_COL__075_573"           
"T_COL__075","A4"."T_COL__076_574" "T_COL__076","A4"."T_COL__077_575"           
"T_COL__077","A4"."T_COL__078_576" "T_COL__078","A4"."T_COL__079_577"           
"T_COL__079","A4"."T_COL__080_578" "T_COL__080","A4"."T_COL__081_579"           
"T_COL__081","A4"."T_COL__082_580" "T_COL__082","A4"."T_COL__083_581"           
"T_COL__083","A4"."T_COL__084_582" "T_COL__084","A4"."T_COL__085_583"           
"T_COL__085","A4"."T_COL__086_584" "T_COL__086","A4"."T_COL__087_585"           
"T_COL__087","A4"."T_COL__088_586" "T_COL__088","A4"."T_COL__089_587"           
"T_COL__089","A4"."T_COL__090_588" "T_COL__090","A4"."T_COL__091_589"           
"T_COL__091","A4"."T_COL__092_590" "T_COL__092","A4"."T_COL__093_591"           
"T_COL__093","A4"."T_COL__094_592" "T_COL__094","A4"."T_COL__095_593"           
"T_COL__095","A4"."T_COL__096_594" "T_COL__096","A4"."T_COL__097_595"           
"T_COL__097","A4"."T_COL__098_596" "T_COL__098","A4"."T_COL__099_597"           
"T_COL__099","A4"."T_COL__100_598" "T_COL__100","A4"."T_COL__101_599"           
"T_COL__101","A4"."T_COL__102_600" "T_CO                                        
                                                                                

The same column list is produced by the ANSI expansion. If, however, a single column is selected from the ANSI join no such ‘unlimited’ expansion occurs:


SQL> 
SQL> --
SQL> -- Selecting only one column from the ANSI join doesn't produce the exaggerated
SQL> -- column list
SQL> --
SQL> 
SQL> set serveroutput on
SQL> set long 20000
SQL> 
SQL> variable m_sql_out clob
SQL> 
SQL> declare
  2      m_sql_in    clob :=
  3                          '
  4                          select p.P_COLUMN_002
  5                          from ORANGE g
  6                          join MAUVE r on g.Z_COL_001 = r.Z_COL_001
  7                          join YELLOW p on g.Z_COL_001 = p.Z_COL_001
  8                          ';
  9  begin
 10  
 11      dbms_utility.expand_sql_text(
 12          m_sql_in,
 13          :m_sql_out
 14      );
 15  
 16  end;
 17  /

PL/SQL procedure successfully completed.

SQL> 
SQL> column m_sql_out wrap word
SQL> print m_sql_out
SELECT "A1"."P_COLUMN_002_3" "P_COLUMN_002" FROM  (SELECT "A3"."Z_COL_001_0"    
"Z_COL_001","A3"."Z_COL_001_1" "Z_COL_001","A2"."Z_COL_001"                     
"Z_COL_001","A2"."P_COLUMN_002" "P_COLUMN_002_3" FROM  (SELECT "A5"."Z_COL_001" 
"Z_COL_001_0","A4"."Z_COL_001" "Z_COL_001_1" FROM "BING"."ORANGE"               
"A5","BING"."MAUVE" "A4" WHERE "A5"."Z_COL_001"="A4"."Z_COL_001")               
"A3","BING"."YELLOW" "A2" WHERE "A3"."Z_COL_001_0"="A2"."Z_COL_001") "A1"       
                                                                                

SQL> 
SQL> drop table ORANGE;

Table dropped.

SQL> drop table MAUVE;

Table dropped.

SQL> drop table YELLOW;

Table dropped.

SQL> 

Using ‘old style’ Oracle join syntax also produces the correct column list on expansion:


SQL> declare
  2      m_sql_in    clob :=
  3                          '
  4                          select p.*
  5                          from ORANGE g,
  6                               MAUVE r,
  7                               YELLOW p
  8                          where g.Z_COL_001 = p.Z_COL_001
  9                          and g.Z_COL_001 = r.Z_COL_001
 10                          ';
 11  begin
 12
 13      dbms_utility.expand_sql_text(
 14          m_sql_in,
 15          :m_sql_out
 16      );
 17
 18  end;
 19  /

PL/SQL procedure successfully completed.

SQL>
SQL> column m_sql_out wrap word
SQL> print m_sql_out
SELECT "A1"."Z_COL_001" "Z_COL_001","A1"."P_COLUMN_002" "P_COLUMN_002" FROM
"SYS"."ORANGE" "A3","SYS"."MAUVE" "A2","SYS"."YELLOW" "A1" WHERE
"A3"."Z_COL_001"="A1"."Z_COL_001" AND "A3"."Z_COL_001"="A2"."Z_COL_001"


SQL>

So this appears to be a bug in the ANSI implementation in 12c; earlier releases don’t exhibit this behavior. Even though this isn’t likely to be a common problem in OLTP systems (it’s probably pretty rare to have tables with 500 or more columns, and probably even rarer to have them joined together) in a data warehouse environment this may be fairly common. Oracle support reports this as a workaround:


alter session set "_fix_control"='17376322:OFF';

If the same example is run after the above statement is executed the error does go away:


SQL> 
SQL> alter session set "_fix_control"='17376322:OFF';

Session altered.

SQL> 
SQL> --
SQL> -- Create 500 column tables
SQL> --
SQL> declare v_sql varchar2(32767);
  2  
  3  begin
  4  
  5    -- create test table ORANGE with 50 columns
  6    v_sql := 'create table ORANGE (Z_COL_001 number(9) not null';
  7    for i in 1 .. 499 loop
  8  	 v_sql := v_sql || ',' || chr(10) || 'Z_COL_' || lpad(to_char(i+1), 3, '0') || ' varchar2(2) null';
  9    end loop;
 10    v_sql := v_sql || ')';
 11    execute immediate v_sql;
 12  
 13    -- create test table MAUVE with 50 columns
 14  
 15    v_sql := 'create table MAUVE (Z_COL_001 number(9) not null';
 16    for i in 1 .. 499 loop
 17  	 v_sql := v_sql || ',' || chr(10) || 'T_COL__' || lpad(to_char(i+1), 3, '0') || ' varchar2(2) null';
 18    end loop;
 19    v_sql := v_sql || ')';
 20    execute immediate v_sql;
 21  
 22  end;
 23  /

PL/SQL procedure successfully completed.

SQL> 
SQL> -- create test table YELLOW with 2 columns
SQL> 
SQL> create table YELLOW (Z_COL_001 number(9) not null, P_COLUMN_002 varchar2(2));

Table created.

SQL> 
SQL> --********************************************************************************--
SQL> -- selecting one column work!
SQL> --********************************************************************************--
SQL> 
SQL> select p.P_COLUMN_002
  2  from ORANGE g
  3    join MAUVE r on g.Z_COL_001 = r.Z_COL_001
  4    join YELLOW p on g.Z_COL_001 = p.Z_COL_001;

no rows selected

SQL> 
SQL> --********************************************************************************--
SQL> -- notice the YELLOW table only has 2 columns, but selecting p.* fails in Oracle 12c:
SQL> -- SQL Error: ORA-01792: maximum number of columns in a table or view is 1000
SQL> --********************************************************************************--
SQL> 
SQL> select p.*
  2  from ORANGE g
  3    join MAUVE r on g.Z_COL_001 = r.Z_COL_001
  4    join YELLOW p on g.Z_COL_001 = p.Z_COL_001;

no rows selected

SQL> 
SQL> --********************************************************************************--
SQL> -- sub selecting only one column also fails in Oracle 12c:
SQL> -- SQL Error: ORA-01792: maximum number of columns in a table or view is 1000
SQL> --********************************************************************************--
SQL> 
SQL> select (select 1 from dual where 1 = 2) as exp_column
  2  from ORANGE g
  3    join MAUVE r on g.Z_COL_001 = r.Z_COL_001
  4    join YELLOW p on g.Z_COL_001 = p.Z_COL_001;

no rows selected

SQL> 
SQL> drop table ORANGE;

Table dropped.

SQL> 
SQL> drop table MAUVE;

Table dropped.

SQL> 
SQL> drop table YELLOW;

Table dropped.

SQL> 
SQL> --
SQL> -- Create 499 column tables so we can see the expansion problem
SQL> --
SQL> declare v_sql varchar2(32767);
  2  
  3  begin
  4  
  5    -- create test table ORANGE with 50 columns
  6    v_sql := 'create table ORANGE (Z_COL_001 number(9) not null';
  7    for i in 1 .. 498 loop
  8  	 v_sql := v_sql || ',' || chr(10) || 'Z_COL_' || lpad(to_char(i+1), 3, '0') || ' varchar2(2) null';
  9    end loop;
 10    v_sql := v_sql || ')';
 11    execute immediate v_sql;
 12  
 13    -- create test table MAUVE with 50 columns
 14  
 15    v_sql := 'create table MAUVE (Z_COL_001 number(9) not null';
 16    for i in 1 .. 498 loop
 17  	 v_sql := v_sql || ',' || chr(10) || 'T_COL__' || lpad(to_char(i+1), 3, '0') || ' varchar2(2) null';
 18    end loop;
 19    v_sql := v_sql || ')';
 20    execute immediate v_sql;
 21  
 22  end;
 23  /

PL/SQL procedure successfully completed.

SQL> 
SQL> -- create test table YELLOW with 2 columns
SQL> 
SQL> create table YELLOW (Z_COL_001 number(9) not null, P_COLUMN_002 varchar2(2));

Table created.

SQL> 
SQL> --********************************************************************************--
SQL> -- selecting one column work!
SQL> --********************************************************************************--
SQL> 
SQL> select p.P_COLUMN_002
  2  from ORANGE g
  3    join MAUVE r on g.Z_COL_001 = r.Z_COL_001
  4    join YELLOW p on g.Z_COL_001 = p.Z_COL_001;

no rows selected

SQL> 
SQL> --********************************************************************************--
SQL> -- notice the YELLOW table only has 2 columns, but selecting p.* fails in Oracle 12c:
SQL> -- when the joined tables have more than 499 columns each
SQL> -- SQL Error: ORA-01792: maximum number of columns in a table or view is 1000
SQL> --********************************************************************************--
SQL> 
SQL> select p.*
  2  from ORANGE g
  3    join MAUVE r on g.Z_COL_001 = r.Z_COL_001
  4    join YELLOW p on g.Z_COL_001 = p.Z_COL_001;

no rows selected

SQL> 
SQL> --********************************************************************************--
SQL> -- Here's why:
SQL> --
SQL> -- When Oracle attempts an ANSI expansion of the code columns multiply in the ANSI
SQL> -- join and somehow appear to be associated with the various tables included in
SQL> -- the join
SQL> --********************************************************************************--
SQL> 
SQL> set serveroutput on
SQL> set long 20000
SQL> 
SQL> variable m_sql_out clob
SQL> 
SQL> declare
  2  	 m_sql_in    clob :=
  3  			     '
  4  			     select p.*
  5  			     from ORANGE g
  6  			     join MAUVE r on g.Z_COL_001 = r.Z_COL_001
  7  			     join YELLOW p on g.Z_COL_001 = p.Z_COL_001
  8  			     ';
  9  begin
 10  
 11  	 dbms_utility.expand_sql_text(
 12  	     m_sql_in,
 13  	     :m_sql_out
 14  	 );
 15  
 16  end;
 17  /

PL/SQL procedure successfully completed.

SQL> 
SQL> set long 1000000
SQL> column m_sql_out wrap word
SQL> print m_sql_out
SELECT "A1"."Z_COL_001_998" "Z_COL_001","A1"."P_COLUMN_002_999" "P_COLUMN_002"  
FROM  (SELECT "A3"."Z_COL_001_0" "Z_COL_001","A3"."Z_COL_002_1"                 
"Z_COL_002","A3"."Z_COL_003_2" "Z_COL_003","A3"."Z_COL_004_3"                   
"Z_COL_004","A3"."Z_COL_005_4" "Z_COL_005","A3"."Z_COL_006_5"                   
"Z_COL_006","A3"."Z_COL_007_6" "Z_COL_007","A3"."Z_COL_008_7"                   
"Z_COL_008","A3"."Z_COL_009_8" "Z_COL_009","A3"."Z_COL_010_9"                   
"Z_COL_010","A3"."Z_COL_011_10" "Z_COL_011","A3"."Z_COL_012_11"                 
"Z_COL_012","A3"."Z_COL_013_12" "Z_COL_013","A3"."Z_COL_014_13"                 
"Z_COL_014","A3"."Z_COL_015_14" "Z_COL_015","A3"."Z_COL_016_15"                 
"Z_COL_016","A3"."Z_COL_017_16" "Z_COL_017","A3"."Z_COL_018_17"                 
"Z_COL_018","A3"."Z_COL_019_18" "Z_COL_019","A3"."Z_COL_020_19"                 
"Z_COL_020","A3"."Z_COL_021_20" "Z_COL_021","A3"."Z_COL_022_21"                 
"Z_COL_022","A3"."Z_COL_023_22" "Z_COL_023","A3"."Z_COL_024_23"                 
"Z_COL_024","A3"."Z_COL_025_24" "Z_COL_025","A3"."Z_COL_026_25"                 
"Z_COL_026","A3"."Z_COL_027_26" "Z_COL_027","A3"."Z_COL_028_27"                 
"Z_COL_028","A3"."Z_COL_029_28" "Z_COL_029","A3"."Z_COL_030_29"                 
"Z_COL_030","A3"."Z_COL_031_30" "Z_COL_031","A3"."Z_COL_032_31"                 
"Z_COL_032","A3"."Z_COL_033_32" "Z_COL_033","A3"."Z_COL_034_33"                 
"Z_COL_034","A3"."Z_COL_035_34" "Z_COL_035","A3"."Z_COL_036_35"                 
"Z_COL_036","A3"."Z_COL_037_36" "Z_COL_037","A3"."Z_COL_038_37"                 
"Z_COL_038","A3"."Z_COL_039_38" "Z_COL_039","A3"."Z_COL_040_39"                 
"Z_COL_040","A3"."Z_COL_041_40" "Z_COL_041","A3"."Z_COL_042_41"                 
"Z_COL_042","A3"."Z_COL_043_42" "Z_COL_043","A3"."Z_COL_044_43"                 
"Z_COL_044","A3"."Z_COL_045_44" "Z_COL_045","A3"."Z_COL_046_45"                 
"Z_COL_046","A3"."Z_COL_047_46" "Z_COL_047","A3"."Z_COL_048_47"                 
"Z_COL_048","A3"."Z_COL_049_48" "Z_COL_049","A3"."Z_COL_050_49"                 
"Z_COL_050","A3"."Z_COL_051_50" "Z_COL_051","A3"."Z_COL_052_51"                 
"Z_COL_052","A3"."Z_COL_053_52" "Z_COL_053","A3"."Z_COL_054_53"                 
"Z_COL_054","A3"."Z_COL_055_54" "Z_COL_055","A3"."Z_COL_056_55"                 
"Z_COL_056","A3"."Z_COL_057_56" "Z_COL_057","A3"."Z_COL_058_57"                 
"Z_COL_058","A3"."Z_COL_059_58" "Z_COL_059","A3"."Z_COL_060_59"                 
"Z_COL_060","A3"."Z_COL_061_60" "Z_COL_061","A3"."Z_COL_062_61"                 
"Z_COL_062","A3"."Z_COL_063_62" "Z_COL_063","A3"."Z_COL_064_63"                 
"Z_COL_064","A3"."Z_COL_065_64" "Z_COL_065","A3"."Z_COL_066_65"                 
"Z_COL_066","A3"."Z_COL_067_66" "Z_COL_067","A3"."Z_COL_068_67"                 
"Z_COL_068","A3"."Z_COL_069_68" "Z_COL_069","A3"."Z_COL_070_69"                 
"Z_COL_070","A3"."Z_COL_071_70" "Z_COL_071","A3"."Z_COL_072_71"                 
"Z_COL_072","A3"."Z_COL_073_72" "Z_COL_073","A3"."Z_COL_074_73"                 
"Z_COL_074","A3"."Z_COL_075_74" "Z_COL_075","A3"."Z_COL_076_75"                 
"Z_COL_076","A3"."Z_COL_077_76" "Z_COL_077","A3"."Z_COL_078_77"                 
"Z_COL_078","A3"."Z_COL_079_78" "Z_COL_079","A3"."Z_COL_080_79"                 
"Z_COL_080","A3"."Z_COL_081_80" "Z_COL_081","A3"."Z_COL_082_81"                 
"Z_COL_082","A3"."Z_COL_083_82" "Z_COL_083","A3"."Z_COL_084_83"                 
"Z_COL_084","A3"."Z_COL_085_84" "Z_COL_085","A3"."Z_COL_086_85"                 
"Z_COL_086","A3"."Z_COL_087_86" "Z_COL_087","A3"."Z_COL_088_87"                 
"Z_COL_088","A3"."Z_COL_089_88" "Z_COL_089","A3"."Z_COL_090_89"                 
"Z_COL_090","A3"."Z_COL_091_90" "Z_COL_091","A3"."Z_COL_092_91"                 
"Z_COL_092","A3"."Z_COL_093_92" "Z_COL_093","A3"."Z_COL_094_93"                 
"Z_COL_094","A3"."Z_COL_095_94" "Z_COL_095","A3"."Z_COL_096_95"                 
"Z_COL_096","A3"."Z_COL_097_96" "Z_COL_097","A3"."Z_COL_098_97"                 
"Z_COL_098","A3"."Z_COL_099_98" "Z_COL_099","A3"."Z_COL_100_99"                 
"Z_COL_100","A3"."Z_COL_101_100" "Z_COL_101","A3"."Z_COL_102_101"               
"Z_COL_102","A3"."Z_COL_103_102" "Z_COL_103","A3"."Z_COL_104_103"               
"Z_COL_104","A3"."Z_COL_105_104" "Z_COL_105","A3"."Z_COL_106_105"               
"Z_COL_106","A3"."Z_COL_107_106" "Z_COL_107","A3"."Z_COL_108_107"               
"Z_COL_108","A3"."Z_COL_109_108" "Z_COL_109","A3"."Z_COL_110_109"               
"Z_COL_110","A3"."Z_COL_111_110" "Z_COL_111","A3"."Z_COL_112_111"               
"Z_COL_112","A3"."Z_COL_113_112" "Z_COL_113","A3"."Z_COL_114_113"               
"Z_COL_114","A3"."Z_COL_115_114" "Z_COL_115","A3"."Z_COL_116_115"               
"Z_COL_116","A3"."Z_COL_117_116" "Z_COL_117","A3"."Z_COL_118_117"               
"Z_COL_118","A3"."Z_COL_119_118" "Z_COL_119","A3"."Z_COL_120_119"               
"Z_COL_120","A3"."Z_COL_121_120" "Z_COL_121","A3"."Z_COL_122_121"               
"Z_COL_122","A3"."Z_COL_123_122" "Z_COL_123","A3"."Z_COL_124_123"               
"Z_COL_124","A3"."Z_COL_125_124" "Z_COL_125","A3"."Z_COL_126_125"               
"Z_COL_126","A3"."Z_COL_127_126" "Z_COL_127","A3"."Z_COL_128_127"               
"Z_COL_128","A3"."Z_COL_129_128" "Z_COL_129","A3"."Z_COL_130_129"               
"Z_COL_130","A3"."Z_COL_131_130" "Z_COL_131","A3"."Z_COL_132_131"               
"Z_COL_132","A3"."Z_COL_133_132" "Z_COL_133","A3"."Z_COL_134_133"               
"Z_COL_134","A3"."Z_COL_135_134" "Z_COL_135","A3"."Z_COL_136_135"               
"Z_COL_136","A3"."Z_COL_137_136" "Z_COL_137","A3"."Z_COL_138_137"               
"Z_COL_138","A3"."Z_COL_139_138" "Z_COL_139","A3"."Z_COL_140_139"               
"Z_COL_140","A3"."Z_COL_141_140" "Z_COL_141","A3"."Z_COL_142_141"               
"Z_COL_142","A3"."Z_COL_143_142" "Z_COL_143","A3"."Z_COL_144_143"               
"Z_COL_144","A3"."Z_COL_145_144" "Z_COL_145","A3"."Z_COL_146_145"               
"Z_COL_146","A3"."Z_COL_147_146" "Z_COL_147","A3"."Z_COL_148_147"               
"Z_COL_148","A3"."Z_COL_149_148" "Z_COL_149","A3"."Z_COL_150_149"               
"Z_COL_150","A3"."Z_COL_151_150" "Z_COL_151","A3"."Z_COL_152_151"               
"Z_COL_152","A3"."Z_COL_153_152" "Z_COL_153","A3"."Z_COL_154_153"               
"Z_COL_154","A3"."Z_COL_155_154" "Z_COL_155","A3"."Z_COL_156_155"               
"Z_COL_156","A3"."Z_COL_157_156" "Z_COL_157","A3"."Z_COL_158_157"               
"Z_COL_158","A3"."Z_COL_159_158" "Z_COL_159","A3"."Z_COL_160_159"               
"Z_COL_160","A3"."Z_COL_161_160" "Z_COL_161","A3"."Z_COL_162_161"               
"Z_COL_162","A3"."Z_COL_163_162" "Z_COL_163","A3"."Z_COL_164_163"               
"Z_COL_164","A3"."Z_COL_165_164" "Z_COL_165","A3"."Z_COL_166_165"               
"Z_COL_166","A3"."Z_COL_167_166" "Z_COL_167","A3"."Z_COL_168_167"               
"Z_COL_168","A3"."Z_COL_169_168" "Z_COL_169","A3"."Z_COL_170_169"               
"Z_COL_170","A3"."Z_COL_171_170" "Z_COL_171","A3"."Z_COL_172_171"               
"Z_COL_172","A3"."Z_COL_173_172" "Z_COL_173","A3"."Z_COL_174_173"               
"Z_COL_174","A3"."Z_COL_175_174" "Z_COL_175","A3"."Z_COL_176_175"               
"Z_COL_176","A3"."Z_COL_177_176" "Z_COL_177","A3"."Z_COL_178_177"               
"Z_COL_178","A3"."Z_COL_179_178" "Z_COL_179","A3"."Z_COL_180_179"               
"Z_COL_180","A3"."Z_COL_181_180" "Z_COL_181","A3"."Z_COL_182_181"               
"Z_COL_182","A3"."Z_COL_183_182" "Z_COL_183","A3"."Z_COL_184_183"               
"Z_COL_184","A3"."Z_COL_185_184" "Z_COL_185","A3"."Z_COL_186_185"               
"Z_COL_186","A3"."Z_COL_187_186" "Z_COL_187","A3"."Z_COL_188_187"               
"Z_COL_188","A3"."Z_COL_189_188" "Z_COL_189","A3"."Z_COL_190_189"               
"Z_COL_190","A3"."Z_COL_191_190" "Z_COL_191","A3"."Z_COL_192_191"               
"Z_COL_192","A3"."Z_COL_193_192" "Z_COL_193","A3"."Z_COL_194_193"               
"Z_COL_194","A3"."Z_COL_195_194" "Z_COL_195","A3"."Z_COL_196_195"               
"Z_COL_196","A3"."Z_COL_197_196" "Z_COL_197","A3"."Z_COL_198_197"               
"Z_COL_198","A3"."Z_COL_199_198" "Z_COL_199","A3"."Z_COL_200_199"               
"Z_COL_200","A3"."Z_COL_201_200" "Z_COL_201","A3"."Z_COL_202_201"               
"Z_COL_202","A3"."Z_COL_203_202" "Z_COL_203","A3"."Z_COL_204_203"               
"Z_COL_204","A3"."Z_COL_205_204" "Z_COL_205","A3"."Z_COL_206_205"               
"Z_COL_206","A3"."Z_COL_207_206" "Z_COL_207","A3"."Z_COL_208_207"               
"Z_COL_208","A3"."Z_COL_209_208" "Z_COL_209","A3"."Z_COL_210_209"               
"Z_COL_210","A3"."Z_COL_211_210" "Z_COL_211","A3"."Z_COL_212_211"               
"Z_COL_212","A3"."Z_COL_213_212" "Z_COL_213","A3"."Z_COL_214_213"               
"Z_COL_214","A3"."Z_COL_215_214" "Z_COL_215","A3"."Z_COL_216_215"               
"Z_COL_216","A3"."Z_COL_217_216" "Z_COL_217","A3"."Z_COL_218_217"               
"Z_COL_218","A3"."Z_COL_219_218" "Z_COL_219","A3"."Z_COL_220_219"               
"Z_COL_220","A3"."Z_COL_221_220" "Z_COL_221","A3"."Z_COL_222_221"               
"Z_COL_222","A3"."Z_COL_223_222" "Z_COL_223","A3"."Z_COL_224_223"               
"Z_COL_224","A3"."Z_COL_225_224" "Z_COL_225","A3"."Z_COL_226_225"               
"Z_COL_226","A3"."Z_COL_227_226" "Z_COL_227","A3"."Z_COL_228_227"               
"Z_COL_228","A3"."Z_COL_229_228" "Z_COL_229","A3"."Z_COL_230_229"               
"Z_COL_230","A3"."Z_COL_231_230" "Z_COL_231","A3"."Z_COL_232_231"               
"Z_COL_232","A3"."Z_COL_233_232" "Z_COL_233","A3"."Z_COL_234_233"               
"Z_COL_234","A3"."Z_COL_235_234" "Z_COL_235","A3"."Z_COL_236_235"               
"Z_COL_236","A3"."Z_COL_237_236" "Z_COL_237","A3"."Z_COL_238_237"               
"Z_COL_238","A3"."Z_COL_239_238" "Z_COL_239","A3"."Z_COL_240_239"               
"Z_COL_240","A3"."Z_COL_241_240" "Z_COL_241","A3"."Z_COL_242_241"               
"Z_COL_242","A3"."Z_COL_243_242" "Z_COL_243","A3"."Z_COL_244_243"               
"Z_COL_244","A3"."Z_COL_245_244" "Z_COL_245","A3"."Z_COL_246_245"               
"Z_COL_246","A3"."Z_COL_247_246" "Z_COL_247","A3"."Z_COL_248_247"               
"Z_COL_248","A3"."Z_COL_249_248" "Z_COL_249","A3"."Z_COL_250_249"               
"Z_COL_250","A3"."Z_COL_251_250" "Z_COL_251","A3"."Z_COL_252_251"               
"Z_COL_252","A3"."Z_COL_253_252" "Z_COL_253","A3"."Z_COL_254_253"               
"Z_COL_254","A3"."Z_COL_255_254" "Z_COL_255","A3"."Z_COL_256_255"               
"Z_COL_256","A3"."Z_COL_257_256" "Z_COL_257","A3"."Z_COL_258_257"               
"Z_COL_258","A3"."Z_COL_259_258" "Z_COL_259","A3"."Z_COL_260_259"               
"Z_COL_260","A3"."Z_COL_261_260" "Z_COL_261","A3"."Z_COL_262_261"               
"Z_COL_262","A3"."Z_COL_263_262" "Z_COL_263","A3"."Z_COL_264_263"               
"Z_COL_264","A3"."Z_COL_265_264" "Z_COL_265","A3"."Z_COL_266_265"               
"Z_COL_266","A3"."Z_COL_267_266" "Z_COL_267","A3"."Z_COL_268_267"               
"Z_COL_268","A3"."Z_COL_269_268" "Z_COL_269","A3"."Z_COL_270_269"               
"Z_COL_270","A3"."Z_COL_271_270" "Z_COL_271","A3"."Z_COL_272_271"               
"Z_COL_272","A3"."Z_COL_273_272" "Z_COL_273","A3"."Z_COL_274_273"               
"Z_COL_274","A3"."Z_COL_275_274" "Z_COL_275","A3"."Z_COL_276_275"               
"Z_COL_276","A3"."Z_COL_277_276" "Z_COL_277","A3"."Z_COL_278_277"               
"Z_COL_278","A3"."Z_COL_279_278" "Z_COL_279","A3"."Z_COL_280_279"               
"Z_COL_280","A3"."Z_COL_281_280" "Z_COL_281","A3"."Z_COL_282_281"               
"Z_COL_282","A3"."Z_COL_283_282" "Z_COL_283","A3"."Z_COL_284_283"               
"Z_COL_284","A3"."Z_COL_285_284" "Z_COL_285","A3"."Z_COL_286_285"               
"Z_COL_286","A3"."Z_COL_287_286" "Z_COL_287","A3"."Z_COL_288_287"               
"Z_COL_288","A3"."Z_COL_289_288" "Z_COL_289","A3"."Z_COL_290_289"               
"Z_COL_290","A3"."Z_COL_291_290" "Z_COL_291","A3"."Z_COL_292_291"               
"Z_COL_292","A3"."Z_COL_293_292" "Z_COL_293","A3"."Z_COL_294_293"               
"Z_COL_294","A3"."Z_COL_295_294" "Z_COL_295","A3"."Z_COL_296_295"               
"Z_COL_296","A3"."Z_COL_297_296" "Z_COL_297","A3"."Z_COL_298_297"               
"Z_COL_298","A3"."Z_COL_299_298" "Z_COL_299","A3"."Z_COL_300_299"               
"Z_COL_300","A3"."Z_COL_301_300" "Z_COL_301","A3"."Z_COL_302_301"               
"Z_COL_302","A3"."Z_COL_303_302" "Z_COL_303","A3"."Z_COL_304_303"               
"Z_COL_304","A3"."Z_COL_305_304" "Z_COL_305","A3"."Z_COL_306_305"               
"Z_COL_306","A3"."Z_COL_307_306" "Z_COL_307","A3"."Z_COL_308_307"               
"Z_COL_308","A3"."Z_COL_309_308" "Z_COL_309","A3"."Z_COL_310_309"               
"Z_COL_310","A3"."Z_COL_311_310" "Z_COL_311","A3"."Z_COL_312_311"               
"Z_COL_312","A3"."Z_COL_313_312" "Z_COL_313","A3"."Z_COL_314_313"               
"Z_COL_314","A3"."Z_COL_315_314" "Z_COL_315","A3"."Z_COL_316_315"               
"Z_COL_316","A3"."Z_COL_317_316" "Z_COL_317","A3"."Z_COL_318_317"               
"Z_COL_318","A3"."Z_COL_319_318" "Z_COL_319","A3"."Z_COL_320_319"               
"Z_COL_320","A3"."Z_COL_321_320" "Z_COL_321","A3"."Z_COL_322_321"               
"Z_COL_322","A3"."Z_COL_323_322" "Z_COL_323","A3"."Z_COL_324_323"               
"Z_COL_324","A3"."Z_COL_325_324" "Z_COL_325","A3"."Z_COL_326_325"               
"Z_COL_326","A3"."Z_COL_327_326" "Z_COL_327","A3"."Z_COL_328_327"               
"Z_COL_328","A3"."Z_COL_329_328" "Z_COL_329","A3"."Z_COL_330_329"               
"Z_COL_330","A3"."Z_COL_331_330" "Z_COL_331","A3"."Z_COL_332_331"               
"Z_COL_332","A3"."Z_COL_333_332" "Z_COL_333","A3"."Z_COL_334_333"               
"Z_COL_334","A3"."Z_COL_335_334" "Z_COL_335","A3"."Z_COL_336_335"               
"Z_COL_336","A3"."Z_COL_337_336" "Z_COL_337","A3"."Z_COL_338_337"               
"Z_COL_338","A3"."Z_COL_339_338" "Z_COL_339","A3"."Z_COL_340_339"               
"Z_COL_340","A3"."Z_COL_341_340" "Z_COL_341","A3"."Z_COL_342_341"               
"Z_COL_342","A3"."Z_COL_343_342" "Z_COL_343","A3"."Z_COL_344_343"               
"Z_COL_344","A3"."Z_COL_345_344" "Z_COL_345","A3"."Z_COL_346_345"               
"Z_COL_346","A3"."Z_COL_347_346" "Z_COL_347","A3"."Z_COL_348_347"               
"Z_COL_348","A3"."Z_COL_349_348" "Z_COL_349","A3"."Z_COL_350_349"               
"Z_COL_350","A3"."Z_COL_351_350" "Z_COL_351","A3"."Z_COL_352_351"               
"Z_COL_352","A3"."Z_COL_353_352" "Z_COL_353","A3"."Z_COL_354_353"               
"Z_COL_354","A3"."Z_COL_355_354" "Z_COL_355","A3"."Z_COL_356_355"               
"Z_COL_356","A3"."Z_COL_357_356" "Z_COL_357","A3"."Z_COL_358_357"               
"Z_COL_358","A3"."Z_COL_359_358" "Z_COL_359","A3"."Z_COL_360_359"               
"Z_COL_360","A3"."Z_COL_361_360" "Z_COL_361","A3"."Z_COL_362_361"               
"Z_COL_362","A3"."Z_COL_363_362" "Z_COL_363","A3"."Z_COL_364_363"               
"Z_COL_364","A3"."Z_COL_365_364" "Z_COL_365","A3"."Z_COL_366_365"               
"Z_COL_366","A3"."Z_COL_367_366" "Z_COL_367","A3"."Z_COL_368_367"               
"Z_COL_368","A3"."Z_COL_369_368" "Z_COL_369","A3"."Z_COL_370_369"               
"Z_COL_370","A3"."Z_COL_371_370" "Z_COL_371","A3"."Z_COL_372_371"               
"Z_COL_372","A3"."Z_COL_373_372" "Z_COL_373","A3"."Z_COL_374_373"               
"Z_COL_374","A3"."Z_COL_375_374" "Z_COL_375","A3"."Z_COL_376_375"               
"Z_COL_376","A3"."Z_COL_377_376" "Z_COL_377","A3"."Z_COL_378_377"               
"Z_COL_378","A3"."Z_COL_379_378" "Z_COL_379","A3"."Z_COL_380_379"               
"Z_COL_380","A3"."Z_COL_381_380" "Z_COL_381","A3"."Z_COL_382_381"               
"Z_COL_382","A3"."Z_COL_383_382" "Z_COL_383","A3"."Z_COL_384_383"               
"Z_COL_384","A3"."Z_COL_385_384" "Z_COL_385","A3"."Z_COL_386_385"               
"Z_COL_386","A3"."Z_COL_387_386" "Z_COL_387","A3"."Z_COL_388_387"               
"Z_COL_388","A3"."Z_COL_389_388" "Z_COL_389","A3"."Z_COL_390_389"               
"Z_COL_390","A3"."Z_COL_391_390" "Z_COL_391","A3"."Z_COL_392_391"               
"Z_COL_392","A3"."Z_COL_393_392" "Z_COL_393","A3"."Z_COL_394_393"               
"Z_COL_394","A3"."Z_COL_395_394" "Z_COL_395","A3"."Z_COL_396_395"               
"Z_COL_396","A3"."Z_COL_397_396" "Z_COL_397","A3"."Z_COL_398_397"               
"Z_COL_398","A3"."Z_COL_399_398" "Z_COL_399","A3"."Z_COL_400_399"               
"Z_COL_400","A3"."Z_COL_401_400" "Z_COL_401","A3"."Z_COL_402_401"               
"Z_COL_402","A3"."Z_COL_403_402" "Z_COL_403","A3"."Z_COL_404_403"               
"Z_COL_404","A3"."Z_COL_405_404" "Z_COL_405","A3"."Z_COL_406_405"               
"Z_COL_406","A3"."Z_COL_407_406" "Z_COL_407","A3"."Z_COL_408_407"               
"Z_COL_408","A3"."Z_COL_409_408" "Z_COL_409","A3"."Z_COL_410_409"               
"Z_COL_410","A3"."Z_COL_411_410" "Z_COL_411","A3"."Z_COL_412_411"               
"Z_COL_412","A3"."Z_COL_413_412" "Z_COL_413","A3"."Z_COL_414_413"               
"Z_COL_414","A3"."Z_COL_415_414" "Z_COL_415","A3"."Z_COL_416_415"               
"Z_COL_416","A3"."Z_COL_417_416" "Z_COL_417","A3"."Z_COL_418_417"               
"Z_COL_418","A3"."Z_COL_419_418" "Z_COL_419","A3"."Z_COL_420_419"               
"Z_COL_420","A3"."Z_COL_421_420" "Z_COL_421","A3"."Z_COL_422_421"               
"Z_COL_422","A3"."Z_COL_423_422" "Z_COL_423","A3"."Z_COL_424_423"               
"Z_COL_424","A3"."Z_COL_425_424" "Z_COL_425","A3"."Z_COL_426_425"               
"Z_COL_426","A3"."Z_COL_427_426" "Z_COL_427","A3"."Z_COL_428_427"               
"Z_COL_428","A3"."Z_COL_429_428" "Z_COL_429","A3"."Z_COL_430_429"               
"Z_COL_430","A3"."Z_COL_431_430" "Z_COL_431","A3"."Z_COL_432_431"               
"Z_COL_432","A3"."Z_COL_433_432" "Z_COL_433","A3"."Z_COL_434_433"               
"Z_COL_434","A3"."Z_COL_435_434" "Z_COL_435","A3"."Z_COL_436_435"               
"Z_COL_436","A3"."Z_COL_437_436" "Z_COL_437","A3"."Z_COL_438_437"               
"Z_COL_438","A3"."Z_COL_439_438" "Z_COL_439","A3"."Z_COL_440_439"               
"Z_COL_440","A3"."Z_COL_441_440" "Z_COL_441","A3"."Z_COL_442_441"               
"Z_COL_442","A3"."Z_COL_443_442" "Z_COL_443","A3"."Z_COL_444_443"               
"Z_COL_444","A3"."Z_COL_445_444" "Z_COL_445","A3"."Z_COL_446_445"               
"Z_COL_446","A3"."Z_COL_447_446" "Z_COL_447","A3"."Z_COL_448_447"               
"Z_COL_448","A3"."Z_COL_449_448" "Z_COL_449","A3"."Z_COL_450_449"               
"Z_COL_450","A3"."Z_COL_451_450" "Z_COL_451","A3"."Z_COL_452_451"               
"Z_COL_452","A3"."Z_COL_453_452" "Z_COL_453","A3"."Z_COL_454_453"               
"Z_COL_454","A3"."Z_COL_455_454" "Z_COL_455","A3"."Z_COL_456_455"               
"Z_COL_456","A3"."Z_COL_457_456" "Z_COL_457","A3"."Z_COL_458_457"               
"Z_COL_458","A3"."Z_COL_459_458" "Z_COL_459","A3"."Z_COL_460_459"               
"Z_COL_460","A3"."Z_COL_461_460" "Z_COL_461","A3"."Z_COL_462_461"               
"Z_COL_462","A3"."Z_COL_463_462" "Z_COL_463","A3"."Z_COL_464_463"               
"Z_COL_464","A3"."Z_COL_465_464" "Z_COL_465","A3"."Z_COL_466_465"               
"Z_COL_466","A3"."Z_COL_467_466" "Z_COL_467","A3"."Z_COL_468_467"               
"Z_COL_468","A3"."Z_COL_469_468" "Z_COL_469","A3"."Z_COL_470_469"               
"Z_COL_470","A3"."Z_COL_471_470" "Z_COL_471","A3"."Z_COL_472_471"               
"Z_COL_472","A3"."Z_COL_473_472" "Z_COL_473","A3"."Z_COL_474_473"               
"Z_COL_474","A3"."Z_COL_475_474" "Z_COL_475","A3"."Z_COL_476_475"               
"Z_COL_476","A3"."Z_COL_477_476" "Z_COL_477","A3"."Z_COL_478_477"               
"Z_COL_478","A3"."Z_COL_479_478" "Z_COL_479","A3"."Z_COL_480_479"               
"Z_COL_480","A3"."Z_COL_481_480" "Z_COL_481","A3"."Z_COL_482_481"               
"Z_COL_482","A3"."Z_COL_483_482" "Z_COL_483","A3"."Z_COL_484_483"               
"Z_COL_484","A3"."Z_COL_485_484" "Z_COL_485","A3"."Z_COL_486_485"               
"Z_COL_486","A3"."Z_COL_487_486" "Z_COL_487","A3"."Z_COL_488_487"               
"Z_COL_488","A3"."Z_COL_489_488" "Z_COL_489","A3"."Z_COL_490_489"               
"Z_COL_490","A3"."Z_COL_491_490" "Z_COL_491","A3"."Z_COL_492_491"               
"Z_COL_492","A3"."Z_COL_493_492" "Z_COL_493","A3"."Z_COL_494_493"               
"Z_COL_494","A3"."Z_COL_495_494" "Z_COL_495","A3"."Z_COL_496_495"               
"Z_COL_496","A3"."Z_COL_497_496" "Z_COL_497","A3"."Z_COL_498_497"               
"Z_COL_498","A3"."Z_COL_499_498" "Z_COL_499","A3"."Z_COL_001_499"               
"Z_COL_001","A3"."T_COL__002_500" "T_COL__002","A3"."T_COL__003_501"            
"T_COL__003","A3"."T_COL__004_502" "T_COL__004","A3"."T_COL__005_503"           
"T_COL__005","A3"."T_COL__006_504" "T_COL__006","A3"."T_COL__007_505"           
"T_COL__007","A3"."T_COL__008_506" "T_COL__008","A3"."T_COL__009_507"           
"T_COL__009","A3"."T_COL__010_508" "T_COL__010","A3"."T_COL__011_509"           
"T_COL__011","A3"."T_COL__012_510" "T_COL__012","A3"."T_COL__013_511"           
"T_COL__013","A3"."T_COL__014_512" "T_COL__014","A3"."T_COL__015_513"           
"T_COL__015","A3"."T_COL__016_514" "T_COL__016","A3"."T_COL__017_515"           
"T_COL__017","A3"."T_COL__018_516" "T_COL__018","A3"."T_COL__019_517"           
"T_COL__019","A3"."T_COL__020_518" "T_COL__020","A3"."T_COL__021_519"           
"T_COL__021","A3"."T_COL__022_520" "T_COL__022","A3"."T_COL__023_521"           
"T_COL__023","A3"."T_COL__024_522" "T_COL__024","A3"."T_COL__025_523"           
"T_COL__025","A3"."T_COL__026_524" "T_COL__026","A3"."T_COL__027_525"           
"T_COL__027","A3"."T_COL__028_526" "T_COL__028","A3"."T_COL__029_527"           
"T_COL__029","A3"."T_COL__030_528" "T_COL__030","A3"."T_COL__031_529"           
"T_COL__031","A3"."T_COL__032_530" "T_COL__032","A3"."T_COL__033_531"           
"T_COL__033","A3"."T_COL__034_532" "T_COL__034","A3"."T_COL__035_533"           
"T_COL__035","A3"."T_COL__036_534" "T_COL__036","A3"."T_COL__037_535"           
"T_COL__037","A3"."T_COL__038_536" "T_COL__038","A3"."T_COL__039_537"           
"T_COL__039","A3"."T_COL__040_538" "T_COL__040","A3"."T_COL__041_539"           
"T_COL__041","A3"."T_COL__042_540" "T_COL__042","A3"."T_COL__043_541"           
"T_COL__043","A3"."T_COL__044_542" "T_COL__044","A3"."T_COL__045_543"           
"T_COL__045","A3"."T_COL__046_544" "T_COL__046","A3"."T_COL__047_545"           
"T_COL__047","A3"."T_COL__048_546" "T_COL__048","A3"."T_COL__049_547"           
"T_COL__049","A3"."T_COL__050_548" "T_COL__050","A3"."T_COL__051_549"           
"T_COL__051","A3"."T_COL__052_550" "T_COL__052","A3"."T_COL__053_551"           
"T_COL__053","A3"."T_COL__054_552" "T_COL__054","A3"."T_COL__055_553"           
"T_COL__055","A3"."T_COL__056_554" "T_COL__056","A3"."T_COL__057_555"           
"T_COL__057","A3"."T_COL__058_556" "T_COL__058","A3"."T_COL__059_557"           
"T_COL__059","A3"."T_COL__060_558" "T_COL__060","A3"."T_COL__061_559"           
"T_COL__061","A3"."T_COL__062_560" "T_COL__062","A3"."T_COL__063_561"           
"T_COL__063","A3"."T_COL__064_562" "T_COL__064","A3"."T_COL__065_563"           
"T_COL__065","A3"."T_COL__066_564" "T_COL__066","A3"."T_COL__067_565"           
"T_COL__067","A3"."T_COL__068_566" "T_COL__068","A3"."T_COL__069_567"           
"T_COL__069","A3"."T_COL__070_568" "T_COL__070","A3"."T_COL__071_569"           
"T_COL__071","A3"."T_COL__072_570" "T_COL__072","A3"."T_COL__073_571"           
"T_COL__073","A3"."T_COL__074_572" "T_COL__074","A3"."T_COL__075_573"           
"T_COL__075","A3"."T_COL__076_574" "T_COL__076","A3"."T_COL__077_575"           
"T_COL__077","A3"."T_COL__078_576" "T_COL__078","A3"."T_COL__079_577"           
"T_COL__079","A3"."T_COL__080_578" "T_COL__080","A3"."T_COL__081_579"           
"T_COL__081","A3"."T_COL__082_580" "T_COL__082","A3"."T_COL__083_581"           
"T_COL__083","A3"."T_COL__084_582" "T_COL__084","A3"."T_COL__085_583"           
"T_COL__085","A3"."T_COL__086_584" "T_COL__086","A3"."T_COL__087_585"           
"T_COL__087","A3"."T_COL__088_586" "T_COL__088","A3"."T_COL__089_587"           
"T_COL__089","A3"."T_COL__090_588" "T_COL__090","A3"."T_COL__091_589"           
"T_COL__091","A3"."T_COL__092_590" "T_COL__092","A3"."T_COL__093_591"           
"T_COL__093","A3"."T_COL__094_592" "T_COL__094","A3"."T_COL__095_593"           
"T_COL__095","A3"."T_COL__096_594" "T_COL__096","A3"."T_COL__097_595"           
"T_COL__097","A3"."T_COL__098_596" "T_COL__098","A3"."T_COL__099_597"           
"T_COL__099","A3"."T_COL__100_598" "T_COL__100","A3"."T_COL__101_599"           
"T_COL__101","A3"."T_COL__102_600" "T_COL__102","A3"."T_COL__103_601"           
"T_COL__103","A3"."T_COL__104_602" "T_COL__104","A3"."T_COL__105_603"           
"T_COL__105","A3"."T_COL__106_604" "T_COL__106","A3"."T_COL__107_605"           
"T_COL__107","A3"."T_COL__108_606" "T_COL__108","A3"."T_COL__109_607"           
"T_COL__109","A3"."T_COL__110_608" "T_COL__110","A3"."T_COL__111_609"           
"T_COL__111","A3"."T_COL__112_610" "T_COL__112","A3"."T_COL__113_611"           
"T_COL__113","A3"."T_COL__114_612" "T_COL__114","A3"."T_COL__115_613"           
"T_COL__115","A3"."T_COL__116_614" "T_COL__116","A3"."T_COL__117_615"           
"T_COL__117","A3"."T_COL__118_616" "T_COL__118","A3"."T_COL__119_617"           
"T_COL__119","A3"."T_COL__120_618" "T_COL__120","A3"."T_COL__121_619"           
"T_COL__121","A3"."T_COL__122_620" "T_COL__122","A3"."T_COL__123_621"           
"T_COL__123","A3"."T_COL__124_622" "T_COL__124","A3"."T_COL__125_623"           
"T_COL__125","A3"."T_COL__126_624" "T_COL__126","A3"."T_COL__127_625"           
"T_COL__127","A3"."T_COL__128_626" "T_COL__128","A3"."T_COL__129_627"           
"T_COL__129","A3"."T_COL__130_628" "T_COL__130","A3"."T_COL__131_629"           
"T_COL__131","A3"."T_COL__132_630" "T_COL__132","A3"."T_COL__133_631"           
"T_COL__133","A3"."T_COL__134_632" "T_COL__134","A3"."T_COL__135_633"           
"T_COL__135","A3"."T_COL__136_634" "T_COL__136","A3"."T_COL__137_635"           
"T_COL__137","A3"."T_COL__138_636" "T_COL__138","A3"."T_COL__139_637"           
"T_COL__139","A3"."T_COL__140_638" "T_COL__140","A3"."T_COL__141_639"           
"T_COL__141","A3"."T_COL__142_640" "T_COL__142","A3"."T_COL__143_641"           
"T_COL__143","A3"."T_COL__144_642" "T_COL__144","A3"."T_COL__145_643"           
"T_COL__145","A3"."T_COL__146_644" "T_COL__146","A3"."T_COL__147_645"           
"T_COL__147","A3"."T_COL__148_646" "T_COL__148","A3"."T_COL__149_647"           
"T_COL__149","A3"."T_COL__150_648" "T_COL__150","A3"."T_COL__151_649"           
"T_COL__151","A3"."T_COL__152_650" "T_COL__152","A3"."T_COL__153_651"           
"T_COL__153","A3"."T_COL__154_652" "T_COL__154","A3"."T_COL__155_653"           
"T_COL__155","A3"."T_COL__156_654" "T_COL__156","A3"."T_COL__157_655"           
"T_COL__157","A3"."T_COL__158_656" "T_COL__158","A3"."T_COL__159_657"           
"T_COL__159","A3"."T_COL__160_658" "T_COL__160","A3"."T_COL__161_659"           
"T_COL__161","A3"."T_COL__162_660" "T_COL__162","A3"."T_COL__163_661"           
"T_COL__163","A3"."T_COL__164_662" "T_COL__164","A3"."T_COL__165_663"           
"T_COL__165","A3"."T_COL__166_664" "T_COL__166","A3"."T_COL__167_665"           
"T_COL__167","A3"."T_COL__168_666" "T_COL__168","A3"."T_COL__169_667"           
"T_COL__169","A3"."T_COL__170_668" "T_COL__170","A3"."T_COL__171_669"           
"T_COL__171","A3"."T_COL__172_670" "T_COL__172","A3"."T_COL__173_671"           
"T_COL__173","A3"."T_COL__174_672" "T_COL__174","A3"."T_COL__175_673"           
"T_COL__175","A3"."T_COL__176_674" "T_COL__176","A3"."T_COL__177_675"           
"T_COL__177","A3"."T_COL__178_676" "T_COL__178","A3"."T_COL__179_677"           
"T_COL__179","A3"."T_COL__180_678" "T_COL__180","A3"."T_COL__181_679"           
"T_COL__181","A3"."T_COL__182_680" "T_COL__182","A3"."T_COL__183_681"           
"T_COL__183","A3"."T_COL__184_682" "T_COL__184","A3"."T_COL__185_683"           
"T_COL__185","A3"."T_COL__186_684" "T_COL__186","A3"."T_COL__187_685"           
"T_COL__187","A3"."T_COL__188_686" "T_COL__188","A3"."T_COL__189_687"           
"T_COL__189","A3"."T_COL__190_688" "T_COL__190","A3"."T_COL__191_689"           
"T_COL__191","A3"."T_COL__192_690" "T_COL__192","A3"."T_COL__193_691"           
"T_COL__193","A3"."T_COL__194_692" "T_COL__194","A3"."T_COL__195_693"           
"T_COL__195","A3"."T_COL__196_694" "T_COL__196","A3"."T_COL__197_695"           
"T_COL__197","A3"."T_COL__198_696" "T_COL__198","A3"."T_COL__199_697"           
"T_COL__199","A3"."T_COL__200_698" "T_COL__200","A3"."T_COL__201_699"           
"T_COL__201","A3"."T_COL__202_700" "T_COL__202","A3"."T_COL__203_701"           
"T_COL__203","A3"."T_COL__204_702" "T_COL__204","A3"."T_COL__205_703"           
"T_COL__205","A3"."T_COL__206_704" "T_COL__206","A3"."T_COL__207_705"           
"T_COL__207","A3"."T_COL__208_706" "T_COL__208","A3"."T_COL__209_707"           
"T_COL__209","A3"."T_COL__210_708" "T_COL__210","A3"."T_COL__211_709"           
"T_COL__211","A3"."T_COL__212_710" "T_COL__212","A3"."T_COL__213_711"           
"T_COL__213","A3"."T_COL__214_712" "T_COL__214","A3"."T_COL__215_713"           
"T_COL__215","A3"."T_COL__216_714" "T_COL__216","A3"."T_COL__217_715"           
"T_COL__217","A3"."T_COL__218_716" "T_COL__218","A3"."T_COL__219_717"           
"T_COL__219","A3"."T_COL__220_718" "T_COL__220","A3"."T_COL__221_719"           
"T_COL__221","A3"."T_COL__222_720" "T_COL__222","A3"."T_COL__223_721"           
"T_COL__223","A3"."T_COL__224_722" "T_COL__224","A3"."T_COL__225_723"           
"T_COL__225","A3"."T_COL__226_724" "T_COL__226","A3"."T_COL__227_725"           
"T_COL__227","A3"."T_COL__228_726" "T_COL__228","A3"."T_COL__229_727"           
"T_COL__229","A3"."T_COL__230_728" "T_COL__230","A3"."T_COL__231_729"           
"T_COL__231","A3"."T_COL__232_730" "T_COL__232","A3"."T_COL__233_731"           
"T_COL__233","A3"."T_COL__234_732" "T_COL__234","A3"."T_COL__235_733"           
"T_COL__235","A3"."T_COL__236_734" "T_COL__236","A3"."T_COL__237_735"           
"T_COL__237","A3"."T_COL__238_736" "T_COL__238","A3"."T_COL__239_737"           
"T_COL__239","A3"."T_COL__240_738" "T_COL__240","A3"."T_COL__241_739"           
"T_COL__241","A3"."T_COL__242_740" "T_COL__242","A3"."T_COL__243_741"           
"T_COL__243","A3"."T_COL__244_742" "T_COL__244","A3"."T_COL__245_743"           
"T_COL__245","A3"."T_COL__246_744" "T_COL__246","A3"."T_COL__247_745"           
"T_COL__247","A3"."T_COL__248_746" "T_COL__248","A3"."T_COL__249_747"           
"T_COL__249","A3"."T_COL__250_748" "T_COL__250","A3"."T_COL__251_749"           
"T_COL__251","A3"."T_COL__252_750" "T_COL__252","A3"."T_COL__253_751"           
"T_COL__253","A3"."T_COL__254_752" "T_COL__254","A3"."T_COL__255_753"           
"T_COL__255","A3"."T_COL__256_754" "T_COL__256","A3"."T_COL__257_755"           
"T_COL__257","A3"."T_COL__258_756" "T_COL__258","A3"."T_COL__259_757"           
"T_COL__259","A3"."T_COL__260_758" "T_COL__260","A3"."T_COL__261_759"           
"T_COL__261","A3"."T_COL__262_760" "T_COL__262","A3"."T_COL__263_761"           
"T_COL__263","A3"."T_COL__264_762" "T_COL__264","A3"."T_COL__265_763"           
"T_COL__265","A3"."T_COL__266_764" "T_COL__266","A3"."T_COL__267_765"           
"T_COL__267","A3"."T_COL__268_766" "T_COL__268","A3"."T_COL__269_767"           
"T_COL__269","A3"."T_COL__270_768" "T_COL__270","A3"."T_COL__271_769"           
"T_COL__271","A3"."T_COL__272_770" "T_COL__272","A3"."T_COL__273_771"           
"T_COL__273","A3"."T_COL__274_772" "T_COL__274","A3"."T_COL__275_773"           
"T_COL__275","A3"."T_COL__276_774" "T_COL__276","A3"."T_COL__277_775"           
"T_COL__277","A3"."T_COL__278_776" "T_COL__278","A3"."T_COL__279_777"           
"T_COL__279","A3"."T_COL__280_778" "T_COL__280","A3"."T_COL__281_779"           
"T_COL__281","A3"."T_COL__282_780" "T_COL__282","A3"."T_COL__283_781"           
"T_COL__283","A3"."T_COL__284_782" "T_COL__284","A3"."T_COL__285_783"           
"T_COL__285","A3"."T_COL__286_784" "T_COL__286","A3"."T_COL__287_785"           
"T_COL__287","A3"."T_COL__288_786" "T_COL__288","A3"."T_COL__289_787"           
"T_COL__289","A3"."T_COL__290_788" "T_COL__290","A3"."T_COL__291_789"           
"T_COL__291","A3"."T_COL__292_790" "T_COL__292","A3"."T_COL__293_791"           
"T_COL__293","A3"."T_COL__294_792" "T_COL__294","A3"."T_COL__295_793"           
"T_COL__295","A3"."T_COL__296_794" "T_COL__296","A3"."T_COL__297_795"           
"T_COL__297","A3"."T_COL__298_796" "T_COL__298","A3"."T_COL__299_797"           
"T_COL__299","A3"."T_COL__300_798" "T_COL__300","A3"."T_COL__301_799"           
"T_COL__301","A3"."T_COL__302_800" "T_COL__302","A3"."T_COL__303_801"           
"T_COL__303","A3"."T_COL__304_802" "T_COL__304","A3"."T_COL__305_803"           
"T_COL__305","A3"."T_COL__306_804" "T_COL__306","A3"."T_COL__307_805"           
"T_COL__307","A3"."T_COL__308_806" "T_COL__308","A3"."T_COL__309_807"           
"T_COL__309","A3"."T_COL__310_808" "T_COL__310","A3"."T_COL__311_809"           
"T_COL__311","A3"."T_COL__312_810" "T_COL__312","A3"."T_COL__313_811"           
"T_COL__313","A3"."T_COL__314_812" "T_COL__314","A3"."T_COL__315_813"           
"T_COL__315","A3"."T_COL__316_814" "T_COL__316","A3"."T_COL__317_815"           
"T_COL__317","A3"."T_COL__318_816" "T_COL__318","A3"."T_COL__319_817"           
"T_COL__319","A3"."T_COL__320_818" "T_COL__320","A3"."T_COL__321_819"           
"T_COL__321","A3"."T_COL__322_820" "T_COL__322","A3"."T_COL__323_821"           
"T_COL__323","A3"."T_COL__324_822" "T_COL__324","A3"."T_COL__325_823"           
"T_COL__325","A3"."T_COL__326_824" "T_COL__326","A3"."T_COL__327_825"           
"T_COL__327","A3"."T_COL__328_826" "T_COL__328","A3"."T_COL__329_827"           
"T_COL__329","A3"."T_COL__330_828" "T_COL__330","A3"."T_COL__331_829"           
"T_COL__331","A3"."T_COL__332_830" "T_COL__332","A3"."T_COL__333_831"           
"T_COL__333","A3"."T_COL__334_832" "T_COL__334","A3"."T_COL__335_833"           
"T_COL__335","A3"."T_COL__336_834" "T_COL__336","A3"."T_COL__337_835"           
"T_COL__337","A3"."T_COL__338_836" "T_COL__338","A3"."T_COL__339_837"           
"T_COL__339","A3"."T_COL__340_838" "T_COL__340","A3"."T_COL__341_839"           
"T_COL__341","A3"."T_COL__342_840" "T_COL__342","A3"."T_COL__343_841"           
"T_COL__343","A3"."T_COL__344_842" "T_COL__344","A3"."T_COL__345_843"           
"T_COL__345","A3"."T_COL__346_844" "T_COL__346","A3"."T_COL__347_845"           
"T_COL__347","A3"."T_COL__348_846" "T_COL__348","A3"."T_COL__349_847"           
"T_COL__349","A3"."T_COL__350_848" "T_COL__350","A3"."T_COL__351_849"           
"T_COL__351","A3"."T_COL__352_850" "T_COL__352","A3"."T_COL__353_851"           
"T_COL__353","A3"."T_COL__354_852" "T_COL__354","A3"."T_COL__355_853"           
"T_COL__355","A3"."T_COL__356_854" "T_COL__356","A3"."T_COL__357_855"           
"T_COL__357","A3"."T_COL__358_856" "T_COL__358","A3"."T_COL__359_857"           
"T_COL__359","A3"."T_COL__360_858" "T_COL__360","A3"."T_COL__361_859"           
"T_COL__361","A3"."T_COL__362_860" "T_COL__362","A3"."T_COL__363_861"           
"T_COL__363","A3"."T_COL__364_862" "T_COL__364","A3"."T_COL__365_863"           
"T_COL__365","A3"."T_COL__366_864" "T_COL__366","A3"."T_COL__367_865"           
"T_COL__367","A3"."T_COL__368_866" "T_COL__368","A3"."T_COL__369_867"           
"T_COL__369","A3"."T_COL__370_868" "T_COL__370","A3"."T_COL__371_869"           
"T_COL__371","A3"."T_COL__372_870" "T_COL__372","A3"."T_COL__373_871"           
"T_COL__373","A3"."T_COL__374_872" "T_COL__374","A3"."T_COL__375_873"           
"T_COL__375","A3"."T_COL__376_874" "T_COL__376","A3"."T_COL__377_875"           
"T_COL__377","A3"."T_COL__378_876" "T_COL__378","A3"."T_COL__379_877"           
"T_COL__379","A3"."T_COL__380_878" "T_COL__380","A3"."T_COL__381_879"           
"T_COL__381","A3"."T_COL__382_880" "T_COL__382","A3"."T_COL__383_881"           
"T_COL__383","A3"."T_COL__384_882" "T_COL__384","A3"."T_COL__385_883"           
"T_COL__385","A3"."T_COL__386_884" "T_COL__386","A3"."T_COL__387_885"           
"T_COL__387","A3"."T_COL__388_886" "T_COL__388","A3"."T_COL__389_887"           
"T_COL__389","A3"."T_COL__390_888" "T_COL__390","A3"."T_COL__391_889"           
"T_COL__391","A3"."T_COL__392_890" "T_COL__392","A3"."T_COL__393_891"           
"T_COL__393","A3"."T_COL__394_892" "T_COL__394","A3"."T_COL__395_893"           
"T_COL__395","A3"."T_COL__396_894" "T_COL__396","A3"."T_COL__397_895"           
"T_COL__397","A3"."T_COL__398_896" "T_COL__398","A3"."T_COL__399_897"           
"T_COL__399","A3"."T_COL__400_898" "T_COL__400","A3"."T_COL__401_899"           
"T_COL__401","A3"."T_COL__402_900" "T_COL__402","A3"."T_COL__403_901"           
"T_COL__403","A3"."T_COL__404_902" "T_COL__404","A3"."T_COL__405_903"           
"T_COL__405","A3"."T_COL__406_904" "T_COL__406","A3"."T_COL__407_905"           
"T_COL__407","A3"."T_COL__408_906" "T_COL__408","A3"."T_COL__409_907"           
"T_COL__409","A3"."T_COL__410_908" "T_COL__410","A3"."T_COL__411_909"           
"T_COL__411","A3"."T_COL__412_910" "T_COL__412","A3"."T_COL__413_911"           
"T_COL__413","A3"."T_COL__414_912" "T_COL__414","A3"."T_COL__415_913"           
"T_COL__415","A3"."T_COL__416_914" "T_COL__416","A3"."T_COL__417_915"           
"T_COL__417","A3"."T_COL__418_916" "T_COL__418","A3"."T_COL__419_917"           
"T_COL__419","A3"."T_COL__420_918" "T_COL__420","A3"."T_COL__421_919"           
"T_COL__421","A3"."T_COL__422_920" "T_COL__422","A3"."T_COL__423_921"           
"T_COL__423","A3"."T_COL__424_922" "T_COL__424","A3"."T_COL__425_923"           
"T_COL__425","A3"."T_COL__426_924" "T_COL__426","A3"."T_COL__427_925"           
"T_COL__427","A3"."T_COL__428_926" "T_COL__428","A3"."T_COL__429_927"           
"T_COL__429","A3"."T_COL__430_928" "T_COL__430","A3"."T_COL__431_929"           
"T_COL__431","A3"."T_COL__432_930" "T_COL__432","A3"."T_COL__433_931"           
"T_COL__433","A3"."T_COL__434_932" "T_COL__434","A3"."T_COL__435_933"           
"T_COL__435","A3"."T_COL__436_934" "T_COL__436","A3"."T_COL__437_935"           
"T_COL__437","A3"."T_COL__438_936" "T_COL__438","A3"."T_COL__439_937"           
"T_COL__439","A3"."T_COL__440_938" "T_COL__440","A3"."T_COL__441_939"           
"T_COL__441","A3"."T_COL__442_940" "T_COL__442","A3"."T_COL__443_941"           
"T_COL__443","A3"."T_COL__444_942" "T_COL__444","A3"."T_COL__445_943"           
"T_COL__445","A3"."T_COL__446_944" "T_COL__446","A3"."T_COL__447_945"           
"T_COL__447","A3"."T_COL__448_946" "T_COL__448","A3"."T_COL__449_947"           
"T_COL__449","A3"."T_COL__450_948" "T_COL__450","A3"."T_COL__451_949"           
"T_COL__451","A3"."T_COL__452_950" "T_COL__452","A3"."T_COL__453_951"           
"T_COL__453","A3"."T_COL__454_952" "T_COL__454","A3"."T_COL__455_953"           
"T_COL__455","A3"."T_COL__456_954" "T_COL__456","A3"."T_COL__457_955"           
"T_COL__457","A3"."T_COL__458_956" "T_COL__458","A3"."T_COL__459_957"           
"T_COL__459","A3"."T_COL__460_958" "T_COL__460","A3"."T_COL__461_959"           
"T_COL__461","A3"."T_COL__462_960" "T_COL__462","A3"."T_COL__463_961"           
"T_COL__463","A3"."T_COL__464_962" "T_COL__464","A3"."T_COL__465_963"           
"T_COL__465","A3"."T_COL__466_964" "T_COL__466","A3"."T_COL__467_965"           
"T_COL__467","A3"."T_COL__468_966" "T_COL__468","A3"."T_COL__469_967"           
"T_COL__469","A3"."T_COL__470_968" "T_COL__470","A3"."T_COL__471_969"           
"T_COL__471","A3"."T_COL__472_970" "T_COL__472","A3"."T_COL__473_971"           
"T_COL__473","A3"."T_COL__474_972" "T_COL__474","A3"."T_COL__475_973"           
"T_COL__475","A3"."T_COL__476_974" "T_COL__476","A3"."T_COL__477_975"           
"T_COL__477","A3"."T_COL__478_976" "T_COL__478","A3"."T_COL__479_977"           
"T_COL__479","A3"."T_COL__480_978" "T_COL__480","A3"."T_COL__481_979"           
"T_COL__481","A3"."T_COL__482_980" "T_COL__482","A3"."T_COL__483_981"           
"T_COL__483","A3"."T_COL__484_982" "T_COL__484","A3"."T_COL__485_983"           
"T_COL__485","A3"."T_COL__486_984" "T_COL__486","A3"."T_COL__487_985"           
"T_COL__487","A3"."T_COL__488_986" "T_COL__488","A3"."T_COL__489_987"           
"T_COL__489","A3"."T_COL__490_988" "T_COL__490","A3"."T_COL__491_989"           
"T_COL__491","A3"."T_COL__492_990" "T_COL__492","A3"."T_COL__493_991"           
"T_COL__493","A3"."T_COL__494_992" "T_COL__494","A3"."T_COL__495_993"           
"T_COL__495","A3"."T_COL__496_994" "T_COL__496","A3"."T_COL__497_995"           
"T_COL__497","A3"."T_COL__498_996" "T_COL__498","A3"."T_COL__499_997"           
"T_COL__499","A2"."Z_COL_001" "Z_COL_001_998","A2"."P_COLUMN_002"               
"P_COLUMN_002_999" FROM  (SELECT "A5"."Z_COL_001" "Z_COL_001_0","A5"."Z_COL_002"
"Z_COL_002_1","A5"."Z_COL_003" "Z_COL_003_2","A5"."Z_COL_004"                   
"Z_COL_004_3","A5"."Z_COL_005" "Z_COL_005_4","A5"."Z_COL_006"                   
"Z_COL_006_5","A5"."Z_COL_007" "Z_COL_007_6","A5"."Z_COL_008"                   
"Z_COL_008_7","A5"."Z_COL_009" "Z_COL_009_8","A5"."Z_COL_010"                   
"Z_COL_010_9","A5"."Z_COL_011" "Z_COL_011_10","A5"."Z_COL_012"                  
"Z_COL_012_11","A5"."Z_COL_013" "Z_COL_013_12","A5"."Z_COL_014"                 
"Z_COL_014_13","A5"."Z_COL_015" "Z_COL_015_14","A5"."Z_COL_016"                 
"Z_COL_016_15","A5"."Z_COL_017" "Z_COL_017_16","A5"."Z_COL_018"                 
"Z_COL_018_17","A5"."Z_COL_019" "Z_COL_019_18","A5"."Z_COL_020"                 
"Z_COL_020_19","A5"."Z_COL_021" "Z_COL_021_20","A5"."Z_COL_022"                 
"Z_COL_022_21","A5"."Z_COL_023" "Z_COL_023_22","A5"."Z_COL_024"                 
"Z_COL_024_23","A5"."Z_COL_025" "Z_COL_025_24","A5"."Z_COL_026"                 
"Z_COL_026_25","A5"."Z_COL_027" "Z_COL_027_26","A5"."Z_COL_028"                 
"Z_COL_028_27","A5"."Z_COL_029" "Z_COL_029_28","A5"."Z_COL_030"                 
"Z_COL_030_29","A5"."Z_COL_031" "Z_COL_031_30","A5"."Z_COL_032"                 
"Z_COL_032_31","A5"."Z_COL_033" "Z_COL_033_32","A5"."Z_COL_034"                 
"Z_COL_034_33","A5"."Z_COL_035" "Z_COL_035_34","A5"."Z_COL_036"                 
"Z_COL_036_35","A5"."Z_COL_037" "Z_COL_037_36","A5"."Z_COL_038"                 
"Z_COL_038_37","A5"."Z_COL_039" "Z_COL_039_38","A5"."Z_COL_040"                 
"Z_COL_040_39","A5"."Z_COL_041" "Z_COL_041_40","A5"."Z_COL_042"                 
"Z_COL_042_41","A5"."Z_COL_043" "Z_COL_043_42","A5"."Z_COL_044"                 
"Z_COL_044_43","A5"."Z_COL_045" "Z_COL_045_44","A5"."Z_COL_046"                 
"Z_COL_046_45","A5"."Z_COL_047" "Z_COL_047_46","A5"."Z_COL_048"                 
"Z_COL_048_47","A5"."Z_COL_049" "Z_COL_049_48","A5"."Z_COL_050"                 
"Z_COL_050_49","A5"."Z_COL_051" "Z_COL_051_50","A5"."Z_COL_052"                 
"Z_COL_052_51","A5"."Z_COL_053" "Z_COL_053_52","A5"."Z_COL_054"                 
"Z_COL_054_53","A5"."Z_COL_055" "Z_COL_055_54","A5"."Z_COL_056"                 
"Z_COL_056_55","A5"."Z_COL_057" "Z_COL_057_56","A5"."Z_COL_058"                 
"Z_COL_058_57","A5"."Z_COL_059" "Z_COL_059_58","A5"."Z_COL_060"                 
"Z_COL_060_59","A5"."Z_COL_061" "Z_COL_061_60","A5"."Z_COL_062"                 
"Z_COL_062_61","A5"."Z_COL_063" "Z_COL_063_62","A5"."Z_COL_064"                 
"Z_COL_064_63","A5"."Z_COL_065" "Z_COL_065_64","A5"."Z_COL_066"                 
"Z_COL_066_65","A5"."Z_COL_067" "Z_COL_067_66","A5"."Z_COL_068"                 
"Z_COL_068_67","A5"."Z_COL_069" "Z_COL_069_68","A5"."Z_COL_070"                 
"Z_COL_070_69","A5"."Z_COL_071" "Z_COL_071_70","A5"."Z_COL_072"                 
"Z_COL_072_71","A5"."Z_COL_073" "Z_COL_073_72","A5"."Z_COL_074"                 
"Z_COL_074_73","A5"."Z_COL_075" "Z_COL_075_74","A5"."Z_COL_076"                 
"Z_COL_076_75","A5"."Z_COL_077" "Z_COL_077_76","A5"."Z_COL_078"                 
"Z_COL_078_77","A5"."Z_COL_079" "Z_COL_079_78","A5"."Z_COL_080"                 
"Z_COL_080_79","A5"."Z_COL_081" "Z_COL_081_80","A5"."Z_COL_082"                 
"Z_COL_082_81","A5"."Z_COL_083" "Z_COL_083_82","A5"."Z_COL_084"                 
"Z_COL_084_83","A5"."Z_COL_085" "Z_COL_085_84","A5"."Z_COL_086"                 
"Z_COL_086_85","A5"."Z_COL_087" "Z_COL_087_86","A5"."Z_COL_088"                 
"Z_COL_088_87","A5"."Z_COL_089" "Z_COL_089_88","A5"."Z_COL_090"                 
"Z_COL_090_89","A5"."Z_COL_091" "Z_COL_091_90","A5"."Z_COL_092"                 
"Z_COL_092_91","A5"."Z_COL_093" "Z_COL_093_92","A5"."Z_COL_094"                 
"Z_COL_094_93","A5"."Z_COL_095" "Z_COL_095_94","A5"."Z_COL_096"                 
"Z_COL_096_95","A5"."Z_COL_097" "Z_COL_097_96","A5"."Z_COL_098"                 
"Z_COL_098_97","A5"."Z_COL_099" "Z_COL_099_98","A5"."Z_COL_100"                 
"Z_COL_100_99","A5"."Z_COL_101" "Z_COL_101_100","A5"."Z_COL_102"                
"Z_COL_102_101","A5"."Z_COL_103" "Z_COL_103_102","A5"."Z_COL_104"               
"Z_COL_104_103","A5"."Z_COL_105" "Z_COL_105_104","A5"."Z_COL_106"               
"Z_COL_106_105","A5"."Z_COL_107" "Z_COL_107_106","A5"."Z_COL_108"               
"Z_COL_108_107","A5"."Z_COL_109" "Z_COL_109_108","A5"."Z_COL_110"               
"Z_COL_110_109","A5"."Z_COL_111" "Z_COL_111_110","A5"."Z_COL_112"               
"Z_COL_112_111","A5"."Z_COL_113" "Z_COL_113_112","A5"."Z_COL_114"               
"Z_COL_114_113","A5"."Z_COL_115" "Z_COL_115_114","A5"."Z_COL_116"               
"Z_COL_116_115","A5"."Z_COL_117" "Z_COL_117_116","A5"."Z_COL_118"               
"Z_COL_118_117","A5"."Z_COL_119" "Z_COL_119_118","A5"."Z_COL_120"               
"Z_COL_120_119","A5"."Z_COL_121" "Z_COL_121_120","A5"."Z_COL_122"               
"Z_COL_122_121","A5"."Z_COL_123" "Z_COL_123_122","A5"."Z_COL_124"               
"Z_COL_124_123","A5"."Z_COL_125" "Z_COL_125_124","A5"."Z_COL_126"               
"Z_COL_126_125","A5"."Z_COL_127" "Z_COL_127_126","A5"."Z_COL_128"               
"Z_COL_128_127","A5"."Z_COL_129" "Z_COL_129_128","A5"."Z_COL_130"               
"Z_COL_130_129","A5"."Z_COL_131" "Z_COL_131_130","A5"."Z_COL_132"               
"Z_COL_132_131","A5"."Z_COL_133" "Z_COL_133_132","A5"."Z_COL_134"               
"Z_COL_134_133","A5"."Z_COL_135" "Z_COL_135_134","A5"."Z_COL_136"               
"Z_COL_136_135","A5"."Z_COL_137" "Z_COL_137_136","A5"."Z_COL_138"               
"Z_COL_138_137","A5"."Z_COL_139" "Z_COL_139_138","A5"."Z_COL_140"               
"Z_COL_140_139","A5"."Z_COL_141" "Z_COL_141_140","A5"."Z_COL_142"               
"Z_COL_142_141","A5"."Z_COL_143" "Z_COL_143_142","A5"."Z_COL_144"               
"Z_COL_144_143","A5"."Z_COL_145" "Z_COL_145_144","A5"."Z_COL_146"               
"Z_COL_146_145","A5"."Z_COL_147" "Z_COL_147_146","A5"."Z_COL_148"               
"Z_COL_148_147","A5"."Z_COL_149" "Z_COL_149_148","A5"."Z_COL_150"               
"Z_COL_150_149","A5"."Z_COL_151" "Z_COL_151_150","A5"."Z_COL_152"               
"Z_COL_152_151","A5"."Z_COL_153" "Z_COL_153_152","A5"."Z_COL_154"               
"Z_COL_154_153","A5"."Z_COL_155" "Z_COL_155_154","A5"."Z_COL_156"               
"Z_COL_156_155","A5"."Z_COL_157" "Z_COL_157_156","A5"."Z_COL_158"               
"Z_COL_158_157","A5"."Z_COL_159" "Z_COL_159_158","A5"."Z_COL_160"               
"Z_COL_160_159","A5"."Z_COL_161" "Z_COL_161_160","A5"."Z_COL_162"               
"Z_COL_162_161","A5"."Z_COL_163" "Z_COL_163_162","A5"."Z_COL_164"               
"Z_COL_164_163","A5"."Z_COL_165" "Z_COL_165_164","A5"."Z_COL_166"               
"Z_COL_166_165","A5"."Z_COL_167" "Z_COL_167_166","A5"."Z_COL_168"               
"Z_COL_168_167","A5"."Z_COL_169" "Z_COL_169_168","A5"."Z_COL_170"               
"Z_COL_170_169","A5"."Z_COL_171" "Z_COL_171_170","A5"."Z_COL_172"               
"Z_COL_172_171","A5"."Z_COL_173" "Z_COL_173_172","A5"."Z_COL_174"               
"Z_COL_174_173","A5"."Z_COL_175" "Z_COL_175_174","A5"."Z_COL_176"               
"Z_COL_176_175","A5"."Z_COL_177" "Z_COL_177_176","A5"."Z_COL_178"               
"Z_COL_178_177","A5"."Z_COL_179" "Z_COL_179_178","A5"."Z_COL_180"               
"Z_COL_180_179","A5"."Z_COL_181" "Z_COL_181_180","A5"."Z_COL_182"               
"Z_COL_182_181","A5"."Z_COL_183" "Z_COL_183_182","A5"."Z_COL_184"               
"Z_COL_184_183","A5"."Z_COL_185" "Z_COL_185_184","A5"."Z_COL_186"               
"Z_COL_186_185","A5"."Z_COL_187" "Z_COL_187_186","A5"."Z_COL_188"               
"Z_COL_188_187","A5"."Z_COL_189" "Z_COL_189_188","A5"."Z_COL_190"               
"Z_COL_190_189","A5"."Z_COL_191" "Z_COL_191_190","A5"."Z_COL_192"               
"Z_COL_192_191","A5"."Z_COL_193" "Z_COL_193_192","A5"."Z_COL_194"               
"Z_COL_194_193","A5"."Z_COL_195" "Z_COL_195_194","A5"."Z_COL_196"               
"Z_COL_196_195","A5"."Z_COL_197" "Z_COL_197_196","A5"."Z_COL_198"               
"Z_COL_198_197","A5"."Z_COL_199" "Z_COL_199_198","A5"."Z_COL_200"               
"Z_COL_200_199","A5"."Z_COL_201" "Z_COL_201_200","A5"."Z_COL_202"               
"Z_COL_202_201","A5"."Z_COL_203" "Z_COL_203_202","A5"."Z_COL_204"               
"Z_COL_204_203","A5"."Z_COL_205" "Z_COL_205_204","A5"."Z_COL_206"               
"Z_COL_206_205","A5"."Z_COL_207" "Z_COL_207_206","A5"."Z_COL_208"               
"Z_COL_208_207","A5"."Z_COL_209" "Z_COL_209_208","A5"."Z_COL_210"               
"Z_COL_210_209","A5"."Z_COL_211" "Z_COL_211_210","A5"."Z_COL_212"               
"Z_COL_212_211","A5"."Z_COL_213" "Z_COL_213_212","A5"."Z_COL_214"               
"Z_COL_214_213","A5"."Z_COL_215" "Z_COL_215_214","A5"."Z_COL_216"               
"Z_COL_216_215","A5"."Z_COL_217" "Z_COL_217_216","A5"."Z_COL_218"               
"Z_COL_218_217","A5"."Z_COL_219" "Z_COL_219_218","A5"."Z_COL_220"               
"Z_COL_220_219","A5"."Z_COL_221" "Z_COL_221_220","A5"."Z_COL_222"               
"Z_COL_222_221","A5"."Z_COL_223" "Z_COL_223_222","A5"."Z_COL_224"               
"Z_COL_224_223","A5"."Z_COL_225" "Z_COL_225_224","A5"."Z_COL_226"               
"Z_COL_226_225","A5"."Z_COL_227" "Z_COL_227_226","A5"."Z_COL_228"               
"Z_COL_228_227","A5"."Z_COL_229" "Z_COL_229_228","A5"."Z_COL_230"               
"Z_COL_230_229","A5"."Z_COL_231" "Z_COL_231_230","A5"."Z_COL_232"               
"Z_COL_232_231","A5"."Z_COL_233" "Z_COL_233_232","A5"."Z_COL_234"               
"Z_COL_234_233","A5"."Z_COL_235" "Z_COL_235_234","A5"."Z_COL_236"               
"Z_COL_236_235","A5"."Z_COL_237" "Z_COL_237_236","A5"."Z_COL_238"               
"Z_COL_238_237","A5"."Z_COL_239" "Z_COL_239_238","A5"."Z_COL_240"               
"Z_COL_240_239","A5"."Z_COL_241" "Z_COL_241_240","A5"."Z_COL_242"               
"Z_COL_242_241","A5"."Z_COL_243" "Z_COL_243_242","A5"."Z_COL_244"               
"Z_COL_244_243","A5"."Z_COL_245" "Z_COL_245_244","A5"."Z_COL_246"               
"Z_COL_246_245","A5"."Z_COL_247" "Z_COL_247_246","A5"."Z_COL_248"               
"Z_COL_248_247","A5"."Z_COL_249" "Z_COL_249_248","A5"."Z_COL_250"               
"Z_COL_250_249","A5"."Z_COL_251" "Z_COL_251_250","A5"."Z_COL_252"               
"Z_COL_252_251","A5"."Z_COL_253" "Z_COL_253_252","A5"."Z_COL_254"               
"Z_COL_254_253","A5"."Z_COL_255" "Z_COL_255_254","A5"."Z_COL_256"               
"Z_COL_256_255","A5"."Z_COL_257" "Z_COL_257_256","A5"."Z_COL_258"               
"Z_COL_258_257","A5"."Z_COL_259" "Z_COL_259_258","A5"."Z_COL_260"               
"Z_COL_260_259","A5"."Z_COL_261" "Z_COL_261_260","A5"."Z_COL_262"               
"Z_COL_262_261","A5"."Z_COL_263" "Z_COL_263_262","A5"."Z_COL_264"               
"Z_COL_264_263","A5"."Z_COL_265" "Z_COL_265_264","A5"."Z_COL_266"               
"Z_COL_266_265","A5"."Z_COL_267" "Z_COL_267_266","A5"."Z_COL_268"               
"Z_COL_268_267","A5"."Z_COL_269" "Z_COL_269_268","A5"."Z_COL_270"               
"Z_COL_270_269","A5"."Z_COL_271" "Z_COL_271_270","A5"."Z_COL_272"               
"Z_COL_272_271","A5"."Z_COL_273" "Z_COL_273_272","A5"."Z_COL_274"               
"Z_COL_274_273","A5"."Z_COL_275" "Z_COL_275_274","A5"."Z_COL_276"               
"Z_COL_276_275","A5"."Z_COL_277" "Z_COL_277_276","A5"."Z_COL_278"               
"Z_COL_278_277","A5"."Z_COL_279" "Z_COL_279_278","A5"."Z_COL_280"               
"Z_COL_280_279","A5"."Z_COL_281" "Z_COL_281_280","A5"."Z_COL_282"               
"Z_COL_282_281","A5"."Z_COL_283" "Z_COL_283_282","A5"."Z_COL_284"               
"Z_COL_284_283","A5"."Z_COL_285" "Z_COL_285_284","A5"."Z_COL_286"               
"Z_COL_286_285","A5"."Z_COL_287" "Z_COL_287_286","A5"."Z_COL_288"               
"Z_COL_288_287","A5"."Z_COL_289" "Z_COL_289_288","A5"."Z_COL_290"               
"Z_COL_290_289","A5"."Z_COL_291" "Z_COL_291_290","A5"."Z_COL_292"               
"Z_COL_292_291","A5"."Z_COL_293" "Z_COL_293_292","A5"."Z_COL_294"               
"Z_COL_294_293","A5"."Z_COL_295" "Z_COL_295_294","A5"."Z_COL_296"               
"Z_COL_296_295","A5"."Z_COL_297" "Z_COL_297_296","A5"."Z_COL_298"               
"Z_COL_298_297","A5"."Z_COL_299" "Z_COL_299_298","A5"."Z_COL_300"               
"Z_COL_300_299","A5"."Z_COL_301" "Z_COL_301_300","A5"."Z_COL_302"               
"Z_COL_302_301","A5"."Z_COL_303" "Z_COL_303_302","A5"."Z_COL_304"               
"Z_COL_304_303","A5"."Z_COL_305" "Z_COL_305_304","A5"."Z_COL_306"               
"Z_COL_306_305","A5"."Z_COL_307" "Z_COL_307_306","A5"."Z_COL_308"               
"Z_COL_308_307","A5"."Z_COL_309" "Z_COL_309_308","A5"."Z_COL_310"               
"Z_COL_310_309","A5"."Z_COL_311" "Z_COL_311_310","A5"."Z_COL_312"               
"Z_COL_312_311","A5"."Z_COL_313" "Z_COL_313_312","A5"."Z_COL_314"               
"Z_COL_314_313","A5"."Z_COL_315" "Z_COL_315_314","A5"."Z_COL_316"               
"Z_COL_316_315","A5"."Z_COL_317" "Z_COL_317_316","A5"."Z_COL_318"               
"Z_COL_318_317","A5"."Z_COL_319" "Z_COL_319_318","A5"."Z_COL_320"               
"Z_COL_320_319","A5"."Z_COL_321" "Z_COL_321_320","A5"."Z_COL_322"               
"Z_COL_322_321","A5"."Z_COL_323" "Z_COL_323_322","A5"."Z_COL_324"               
"Z_COL_324_323","A5"."Z_COL_325" "Z_COL_325_324","A5"."Z_COL_326"               
"Z_COL_326_325","A5"."Z_COL_327" "Z_COL_327_326","A5"."Z_COL_328"               
"Z_COL_328_327","A5"."Z_COL_329" "Z_COL_329_328","A5"."Z_COL_330"               
"Z_COL_330_329","A5"."Z_COL_331" "Z_COL_331_330","A5"."Z_COL_332"               
"Z_COL_332_331","A5"."Z_COL_333" "Z_COL_333_332","A5"."Z_COL_334"               
"Z_COL_334_333","A5"."Z_COL_335" "Z_COL_335_334","A5"."Z_COL_336"               
"Z_COL_336_335","A5"."Z_COL_337" "Z_COL_337_336","A5"."Z_COL_338"               
"Z_COL_338_337","A5"."Z_COL_339" "Z_COL_339_338","A5"."Z_COL_340"               
"Z_COL_340_339","A5"."Z_COL_341" "Z_COL_341_340","A5"."Z_COL_342"               
"Z_COL_342_341","A5"."Z_COL_343" "Z_COL_343_342","A5"."Z_COL_344"               
"Z_COL_344_343","A5"."Z_COL_345" "Z_COL_345_344","A5"."Z_COL_346"               
"Z_COL_346_345","A5"."Z_COL_347" "Z_COL_347_346","A5"."Z_COL_348"               
"Z_COL_348_347","A5"."Z_COL_349" "Z_COL_349_348","A5"."Z_COL_350"               
"Z_COL_350_349","A5"."Z_COL_351" "Z_COL_351_350","A5"."Z_COL_352"               
"Z_COL_352_351","A5"."Z_COL_353" "Z_COL_353_352","A5"."Z_COL_354"               
"Z_COL_354_353","A5"."Z_COL_355" "Z_COL_355_354","A5"."Z_COL_356"               
"Z_COL_356_355","A5"."Z_COL_357" "Z_COL_357_356","A5"."Z_COL_358"               
"Z_COL_358_357","A5"."Z_COL_359" "Z_COL_359_358","A5"."Z_COL_360"               
"Z_COL_360_359","A5"."Z_COL_361" "Z_COL_361_360","A5"."Z_COL_362"               
"Z_COL_362_361","A5"."Z_COL_363" "Z_COL_363_362","A5"."Z_COL_364"               
"Z_COL_364_363","A5"."Z_COL_365" "Z_COL_365_364","A5"."Z_COL_366"               
"Z_COL_366_365","A5"."Z_COL_367" "Z_COL_367_366","A5"."Z_COL_368"               
"Z_COL_368_367","A5"."Z_COL_369" "Z_COL_369_368","A5"."Z_COL_370"               
"Z_COL_370_369","A5"."Z_COL_371" "Z_COL_371_370","A5"."Z_COL_372"               
"Z_COL_372_371","A5"."Z_COL_373" "Z_COL_373_372","A5"."Z_COL_374"               
"Z_COL_374_373","A5"."Z_COL_375" "Z_COL_375_374","A5"."Z_COL_376"               
"Z_COL_376_375","A5"."Z_COL_377" "Z_COL_377_376","A5"."Z_COL_378"               
"Z_COL_378_377","A5"."Z_COL_379" "Z_COL_379_378","A5"."Z_COL_380"               
"Z_COL_380_379","A5"."Z_COL_381" "Z_COL_381_380","A5"."Z_COL_382"               
"Z_COL_382_381","A5"."Z_COL_383" "Z_COL_383_382","A5"."Z_COL_384"               
"Z_COL_384_383","A5"."Z_COL_385" "Z_COL_385_384","A5"."Z_COL_386"               
"Z_COL_386_385","A5"."Z_COL_387" "Z_COL_387_386","A5"."Z_COL_388"               
"Z_COL_388_387","A5"."Z_COL_389" "Z_COL_389_388","A5"."Z_COL_390"               
"Z_COL_390_389","A5"."Z_COL_391" "Z_COL_391_390","A5"."Z_COL_392"               
"Z_COL_392_391","A5"."Z_COL_393" "Z_COL_393_392","A5"."Z_COL_394"               
"Z_COL_394_393","A5"."Z_COL_395" "Z_COL_395_394","A5"."Z_COL_396"               
"Z_COL_396_395","A5"."Z_COL_397" "Z_COL_397_396","A5"."Z_COL_398"               
"Z_COL_398_397","A5"."Z_COL_399" "Z_COL_399_398","A5"."Z_COL_400"               
"Z_COL_400_399","A5"."Z_COL_401" "Z_COL_401_400","A5"."Z_COL_402"               
"Z_COL_402_401","A5"."Z_COL_403" "Z_COL_403_402","A5"."Z_COL_404"               
"Z_COL_404_403","A5"."Z_COL_405" "Z_COL_405_404","A5"."Z_COL_406"               
"Z_COL_406_405","A5"."Z_COL_407" "Z_COL_407_406","A5"."Z_COL_408"               
"Z_COL_408_407","A5"."Z_COL_409" "Z_COL_409_408","A5"."Z_COL_410"               
"Z_COL_410_409","A5"."Z_COL_411" "Z_COL_411_410","A5"."Z_COL_412"               
"Z_COL_412_411","A5"."Z_COL_413" "Z_COL_413_412","A5"."Z_COL_414"               
"Z_COL_414_413","A5"."Z_COL_415" "Z_COL_415_414","A5"."Z_COL_416"               
"Z_COL_416_415","A5"."Z_COL_417" "Z_COL_417_416","A5"."Z_COL_418"               
"Z_COL_418_417","A5"."Z_COL_419" "Z_COL_419_418","A5"."Z_COL_420"               
"Z_COL_420_419","A5"."Z_COL_421" "Z_COL_421_420","A5"."Z_COL_422"               
"Z_COL_422_421","A5"."Z_COL_423" "Z_COL_423_422","A5"."Z_COL_424"               
"Z_COL_424_423","A5"."Z_COL_425" "Z_COL_425_424","A5"."Z_COL_426"               
"Z_COL_426_425","A5"."Z_COL_427" "Z_COL_427_426","A5"."Z_COL_428"               
"Z_COL_428_427","A5"."Z_COL_429" "Z_COL_429_428","A5"."Z_COL_430"               
"Z_COL_430_429","A5"."Z_COL_431" "Z_COL_431_430","A5"."Z_COL_432"               
"Z_COL_432_431","A5"."Z_COL_433" "Z_COL_433_432","A5"."Z_COL_434"               
"Z_COL_434_433","A5"."Z_COL_435" "Z_COL_435_434","A5"."Z_COL_436"               
"Z_COL_436_435","A5"."Z_COL_437" "Z_COL_437_436","A5"."Z_COL_438"               
"Z_COL_438_437","A5"."Z_COL_439" "Z_COL_439_438","A5"."Z_COL_440"               
"Z_COL_440_439","A5"."Z_COL_441" "Z_COL_441_440","A5"."Z_COL_442"               
"Z_COL_442_441","A5"."Z_COL_443" "Z_COL_443_442","A5"."Z_COL_444"               
"Z_COL_444_443","A5"."Z_COL_445" "Z_COL_445_444","A5"."Z_COL_446"               
"Z_COL_446_445","A5"."Z_COL_447" "Z_COL_447_446","A5"."Z_COL_448"               
"Z_COL_448_447","A5"."Z_COL_449" "Z_COL_449_448","A5"."Z_COL_450"               
"Z_COL_450_449","A5"."Z_COL_451" "Z_COL_451_450","A5"."Z_COL_452"               
"Z_COL_452_451","A5"."Z_COL_453" "Z_COL_453_452","A5"."Z_COL_454"               
"Z_COL_454_453","A5"."Z_COL_455" "Z_COL_455_454","A5"."Z_COL_456"               
"Z_COL_456_455","A5"."Z_COL_457" "Z_COL_457_456","A5"."Z_COL_458"               
"Z_COL_458_457","A5"."Z_COL_459" "Z_COL_459_458","A5"."Z_COL_460"               
"Z_COL_460_459","A5"."Z_COL_461" "Z_COL_461_460","A5"."Z_COL_462"               
"Z_COL_462_461","A5"."Z_COL_463" "Z_COL_463_462","A5"."Z_COL_464"               
"Z_COL_464_463","A5"."Z_COL_465" "Z_COL_465_464","A5"."Z_COL_466"               
"Z_COL_466_465","A5"."Z_COL_467" "Z_COL_467_466","A5"."Z_COL_468"               
"Z_COL_468_467","A5"."Z_COL_469" "Z_COL_469_468","A5"."Z_COL_470"               
"Z_COL_470_469","A5"."Z_COL_471" "Z_COL_471_470","A5"."Z_COL_472"               
"Z_COL_472_471","A5"."Z_COL_473" "Z_COL_473_472","A5"."Z_COL_474"               
"Z_COL_474_473","A5"."Z_COL_475" "Z_COL_475_474","A5"."Z_COL_476"               
"Z_COL_476_475","A5"."Z_COL_477" "Z_COL_477_476","A5"."Z_COL_478"               
"Z_COL_478_477","A5"."Z_COL_479" "Z_COL_479_478","A5"."Z_COL_480"               
"Z_COL_480_479","A5"."Z_COL_481" "Z_COL_481_480","A5"."Z_COL_482"               
"Z_COL_482_481","A5"."Z_COL_483" "Z_COL_483_482","A5"."Z_COL_484"               
"Z_COL_484_483","A5"."Z_COL_485" "Z_COL_485_484","A5"."Z_COL_486"               
"Z_COL_486_485","A5"."Z_COL_487" "Z_COL_487_486","A5"."Z_COL_488"               
"Z_COL_488_487","A5"."Z_COL_489" "Z_COL_489_488","A5"."Z_COL_490"               
"Z_COL_490_489","A5"."Z_COL_491" "Z_COL_491_490","A5"."Z_COL_492"               
"Z_COL_492_491","A5"."Z_COL_493" "Z_COL_493_492","A5"."Z_COL_494"               
"Z_COL_494_493","A5"."Z_COL_495" "Z_COL_495_494","A5"."Z_COL_496"               
"Z_COL_496_495","A5"."Z_COL_497" "Z_COL_497_496","A5"."Z_COL_498"               
"Z_COL_498_497","A5"."Z_COL_499" "Z_COL_499_498","A4"."Z_COL_001"               
"Z_COL_001_499","A4"."T_COL__002" "T_COL__002_500","A4"."T_COL__003"            
"T_COL__003_501","A4"."T_COL__004" "T_COL__004_502","A4"."T_COL__005"           
"T_COL__005_503","A4"."T_COL__006" "T_COL__006_504","A4"."T_COL__007"           
"T_COL__007_505","A4"."T_COL__008" "T_COL__008_506","A4"."T_COL__009"           
"T_COL__009_507","A4"."T_COL__010" "T_COL__010_508","A4"."T_COL__011"           
"T_COL__011_509","A4"."T_COL__012" "T_COL__012_510","A4"."T_COL__013"           
"T_COL__013_511","A4"."T_COL__014" "T_COL__014_512","A4"."T_COL__015"           
"T_COL__015_513","A4"."T_COL__016" "T_COL__016_514","A4"."T_COL__017"           
"T_COL__017_515","A4"."T_COL__018" "T_COL__018_516","A4"."T_COL__019"           
"T_COL__019_517","A4"."T_COL__020" "T_COL__020_518","A4"."T_COL__021"           
"T_COL__021_519","A4"."T_COL__022" "T_COL__022_520","A4"."T_COL__023"           
"T_COL__023_521","A4"."T_COL__024" "T_COL__024_522","A4"."T_COL__025"           
"T_COL__025_523","A4"."T_COL__026" "T_COL__026_524","A4"."T_COL__027"           
"T_COL__027_525","A4"."T_COL__028" "T_COL__028_526","A4"."T_COL__029"           
"T_COL__029_527","A4"."T_COL__030" "T_COL__030_528","A4"."T_COL__031"           
"T_COL__031_529","A4"."T_COL__032" "T_COL__032_530","A4"."T_COL__033"           
"T_COL__033_531","A4"."T_COL__034" "T_COL__034_532","A4"."T_COL__035"           
"T_COL__035_533","A4"."T_COL__036" "T_COL__036_534","A4"."T_COL__037"           
"T_COL__037_535","A4"."T_COL__038" "T_COL__038_536","A4"."T_COL__039"           
"T_COL__039_537","A4"."T_COL__040" "T_COL__040_538","A4"."T_COL__041"           
"T_COL__041_539","A4"."T_COL__042" "T_COL__042_540","A4"."T_COL__043"           
"T_COL__043_541","A4"."T_COL__044" "T_COL__044_542","A4"."T_COL__045"           
"T_COL__045_543","A4"."T_COL__046" "T_COL__046_544","A4"."T_COL__047"           
"T_COL__047_545","A4"."T_COL__048" "T_COL__048_546","A4"."T_COL__049"           
"T_COL__049_547","A4"."T_COL__050" "T_COL__050_548","A4"."T_COL__051"           
"T_COL__051_549","A4"."T_COL__052" "T_COL__052_550","A4"."T_COL__053"           
"T_COL__053_551","A4"."T_COL__054" "T_COL__054_552","A4"."T_COL__055"           
"T_COL__055_553","A4"."T_COL__056" "T_COL__056_554","A4"."T_COL__057"           
"T_COL__057_555","A4"."T_COL__058" "T_COL__058_556","A4"."T_COL__059"           
"T_COL__059_557","A4"."T_COL__060" "T_COL__060_558","A4"."T_COL__061"           
"T_COL__061_559","A4"."T_COL__062" "T_COL__062_560","A4"."T_COL__063"           
"T_COL__063_561","A4"."T_COL__064" "T_COL__064_562","A4"."T_COL__065"           
"T_COL__065_563","A4"."T_COL__066" "T_COL__066_564","A4"."T_COL__067"           
"T_COL__067_565","A4"."T_COL__068" "T_COL__068_566","A4"."T_COL__069"           
"T_COL__069_567","A4"."T_COL__070" "T_COL__070_568","A4"."T_COL__071"           
"T_COL__071_569","A4"."T_COL__072" "T_COL__072_570","A4"."T_COL__073"           
"T_COL__073_571","A4"."T_COL__074" "T_COL__074_572","A4"."T_COL__075"           
"T_COL__075_573","A4"."T_COL__076" "T_COL__076_574","A4"."T_COL__077"           
"T_COL__077_575","A4"."T_COL__078" "T_COL__078_576","A4"."T_COL__079"           
"T_COL__079_577","A4"."T_COL__080" "T_COL__080_578","A4"."T_COL__081"           
"T_COL__081_579","A4"."T_COL__082" "T_COL__082_580","A4"."T_COL__083"           
"T_COL__083_581","A4"."T_COL__084" "T_COL__084_582","A4"."T_COL__085"           
"T_COL__085_583","A4"."T_COL__086" "T_COL__086_584","A4"."T_COL__087"           
"T_COL__087_585","A4"."T_COL__088" "T_COL__088_586","A4"."T_COL__089"           
"T_COL__089_587","A4"."T_COL__090" "T_COL__090_588","A4"."T_COL__091"           
"T_COL__091_589","A4"."T_COL__092" "T_COL__092_590","A4"."T_COL__093"           
"T_COL__093_591","A4"."T_COL__094" "T_COL__094_592","A4"."T_COL__095"           
"T_COL__095_593","A4"."T_COL__096" "T_COL__096_594","A4"."T_COL__097"           
"T_COL__097_595","A4"."T_COL__098" "T_COL__098_596","A4"."T_COL__099"           
"T_COL__099_597","A4"."T_COL__100" "T_COL__100_598","A4"."T_COL__101"           
"T_COL__101_599","A4"."T_COL__102" "T_COL__102_600","A4"."T_COL__103"           
"T_COL__103_601","A4"."T_COL__104" "T_COL__104_602","A4"."T_COL__105"           
"T_COL__105_603","A4"."T_COL__106" "T_COL__106_604","A4"."T_COL__107"           
"T_COL__107_605","A4"."T_COL__108" "T_COL__108_606","A4"."T_COL__109"           
"T_COL__109_607","A4"."T_COL__110" "T_COL__110_608","A4"."T_COL__111"           
"T_COL__111_609","A4"."T_COL__112" "T_COL__112_610","A4"."T_COL__113"           
"T_COL__113_611","A4"."T_COL__114" "T_COL__114_612","A4"."T_COL__115"           
"T_COL__115_613","A4"."T_COL__116" "T_COL__116_614","A4"."T_COL__117"           
"T_COL__117_615","A4"."T_COL__118" "T_COL__118_616","A4"."T_COL__119"           
"T_COL__119_617","A4"."T_COL__120" "T_COL__120_618","A4"."T_COL__121"           
"T_COL__121_619","A4"."T_COL__122" "T_COL__122_620","A4"."T_COL__123"           
"T_COL__123_621","A4"."T_COL__124" "T_COL__124_622","A4"."T_COL__125"           
"T_COL__125_623","A4"."T_COL__126" "T_COL__126_624","A4"."T_COL__127"           
"T_COL__127_625","A4"."T_COL__128" "T_COL__128_626","A4"."T_COL__129"           
"T_COL__129_627","A4"."T_COL__130" "T_COL__130_628","A4"."T_COL__131"           
"T_COL__131_629","A4"."T_COL__132" "T_COL__132_630","A4"."T_COL__133"           
"T_COL__133_631","A4"."T_COL__134" "T_COL__134_632","A4"."T_COL__135"           
"T_COL__135_633","A4"."T_COL__136" "T_COL__136_634","A4"."T_COL__137"           
"T_COL__137_635","A4"."T_COL__138" "T_COL__138_636","A4"."T_COL__139"           
"T_COL__139_637","A4"."T_COL__140" "T_COL__140_638","A4"."T_COL__141"           
"T_COL__141_639","A4"."T_COL__142" "T_COL__142_640","A4"."T_COL__143"           
"T_COL__143_641","A4"."T_COL__144" "T_COL__144_642","A4"."T_COL__145"           
"T_COL__145_643","A4"."T_COL__146" "T_COL__146_644","A4"."T_COL__147"           
"T_COL__147_645","A4"."T_COL__148" "T_COL__148_646","A4"."T_COL__149"           
"T_COL__149_647","A4"."T_COL__150" "T_COL__150_648","A4"."T_COL__151"           
"T_COL__151_649","A4"."T_COL__152" "T_COL__152_650","A4"."T_COL__153"           
"T_COL__153_651","A4"."T_COL__154" "T_COL__154_652","A4"."T_COL__155"           
"T_COL__155_653","A4"."T_COL__156" "T_COL__156_654","A4"."T_COL__157"           
"T_COL__157_655","A4"."T_COL__158" "T_COL__158_656","A4"."T_COL__159"           
"T_COL__159_657","A4"."T_COL__160" "T_COL__160_658","A4"."T_COL__161"           
"T_COL__161_659","A4"."T_COL__162" "T_COL__162_660","A4"."T_COL__163"           
"T_COL__163_661","A4"."T_COL__164" "T_COL__164_662","A4"."T_COL__165"           
"T_COL__165_663","A4"."T_COL__166" "T_COL__166_664","A4"."T_COL__167"           
"T_COL__167_665","A4"."T_COL__168" "T_COL__168_666","A4"."T_COL__169"           
"T_COL__169_667","A4"."T_COL__170" "T_COL__170_668","A4"."T_COL__171"           
"T_COL__171_669","A4"."T_COL__172" "T_COL__172_670","A4"."T_COL__173"           
"T_COL__173_671","A4"."T_COL__174" "T_COL__174_672","A4"."T_COL__175"           
"T_COL__175_673","A4"."T_COL__176" "T_COL__176_674","A4"."T_COL__177"           
"T_COL__177_675","A4"."T_COL__178" "T_COL__178_676","A4"."T_COL__179"           
"T_COL__179_677","A4"."T_COL__180" "T_COL__180_678","A4"."T_COL__181"           
"T_COL__181_679","A4"."T_COL__182" "T_COL__182_680","A4"."T_COL__183"           
"T_COL__183_681","A4"."T_COL__184" "T_COL__184_682","A4"."T_COL__185"           
"T_COL__185_683","A4"."T_COL__186" "T_COL__186_684","A4"."T_COL__187"           
"T_COL__187_685","A4"."T_COL__188" "T_COL__188_686","A4"."T_COL__189"           
"T_COL__189_687","A4"."T_COL__190" "T_COL__190_688","A4"."T_COL__191"           
"T_COL__191_689","A4"."T_COL__192" "T_COL__192_690","A4"."T_COL__193"           
"T_COL__193_691","A4"."T_COL__194" "T_COL__194_692","A4"."T_COL__195"           
"T_COL__195_693","A4"."T_COL__196" "T_COL__196_694","A4"."T_COL__197"           
"T_COL__197_695","A4"."T_COL__198" "T_COL__198_696","A4"."T_COL__199"           
"T_COL__199_697","A4"."T_COL__200" "T_COL__200_698","A4"."T_COL__201"           
"T_COL__201_699","A4"."T_COL__202" "T_COL__202_700","A4"."T_COL__203"           
"T_COL__203_701","A4"."T_COL__204" "T_COL__204_702","A4"."T_COL__205"           
"T_COL__205_703","A4"."T_COL__206" "T_COL__206_704","A4"."T_COL__207"           
"T_COL__207_705","A4"."T_COL__208" "T_COL__208_706","A4"."T_COL__209"           
"T_COL__209_707","A4"."T_COL__210" "T_COL__210_708","A4"."T_COL__211"           
"T_COL__211_709","A4"."T_COL__212" "T_COL__212_710","A4"."T_COL__213"           
"T_COL__213_711","A4"."T_COL__214" "T_COL__214_712","A4"."T_COL__215"           
"T_COL__215_713","A4"."T_COL__216" "T_COL__216_714","A4"."T_COL__217"           
"T_COL__217_715","A4"."T_COL__218" "T_COL__218_716","A4"."T_COL__219"           
"T_COL__219_717","A4"."T_COL__220" "T_COL__220_718","A4"."T_COL__221"           
"T_COL__221_719","A4"."T_COL__222" "T_COL__222_720","A4"."T_COL__223"           
"T_COL__223_721","A4"."T_COL__224" "T_COL__224_722","A4"."T_COL__225"           
"T_COL__225_723","A4"."T_COL__226" "T_COL__226_724","A4"."T_COL__227"           
"T_COL__227_725","A4"."T_COL__228" "T_COL__228_726","A4"."T_COL__229"           
"T_COL__229_727","A4"."T_COL__230" "T_COL__230_728","A4"."T_COL__231"           
"T_COL__231_729","A4"."T_COL__232" "T_COL__232_730","A4"."T_COL__233"           
"T_COL__233_731","A4"."T_COL__234" "T_COL__234_732","A4"."T_COL__235"           
"T_COL__235_733","A4"."T_COL__236" "T_COL__236_734","A4"."T_COL__237"           
"T_COL__237_735","A4"."T_COL__238" "T_COL__238_736","A4"."T_COL__239"           
"T_COL__239_737","A4"."T_COL__240" "T_COL__240_738","A4"."T_COL__241"           
"T_COL__241_739","A4"."T_COL__242" "T_COL__242_740","A4"."T_COL__243"           
"T_COL__243_741","A4"."T_COL__244" "T_COL__244_742","A4"."T_COL__245"           
"T_COL__245_743","A4"."T_COL__246" "T_COL__246_744","A4"."T_COL__247"           
"T_COL__247_745","A4"."T_COL__248" "T_COL__248_746","A4"."T_COL__249"           
"T_COL__249_747","A4"."T_COL__250" "T_COL__250_748","A4"."T_COL__251"           
"T_COL__251_749","A4"."T_COL__252" "T_COL__252_750","A4"."T_COL__253"           
"T_COL__253_751","A4"."T_COL__254" "T_COL__254_752","A4"."T_COL__255"           
"T_COL__255_753","A4"."T_COL__256" "T_COL__256_754","A4"."T_COL__257"           
"T_COL__257_755","A4"."T_COL__258" "T_COL__258_756","A4"."T_COL__259"           
"T_COL__259_757","A4"."T_COL__260" "T_COL__260_758","A4"."T_COL__261"           
"T_COL__261_759","A4"."T_COL__262" "T_COL__262_760","A4"."T_COL__263"           
"T_COL__263_761","A4"."T_COL__264" "T_COL__264_762","A4"."T_COL__265"           
"T_COL__265_763","A4"."T_COL__266" "T_COL__266_764","A4"."T_COL__267"           
"T_COL__267_765","A4"."T_COL__268" "T_COL__268_766","A4"."T_COL__269"           
"T_COL__269_767","A4"."T_COL__270" "T_COL__270_768","A4"."T_COL__271"           
"T_COL__271_769","A4"."T_COL__272" "T_COL__272_770","A4"."T_COL__273"           
"T_COL__273_771","A4"."T_COL__274" "T_COL__274_772","A4"."T_COL__275"           
"T_COL__275_773","A4"."T_COL__276" "T_COL__276_774","A4"."T_COL__277"           
"T_COL__277_775","A4"."T_COL__278" "T_COL__278_776","A4"."T_COL__279"           
"T_COL__279_777","A4"."T_COL__280" "T_COL__280_778","A4"."T_COL__281"           
"T_COL__281_779","A4"."T_COL__282" "T_COL__282_780","A4"."T_COL__283"           
"T_COL__283_781","A4"."T_COL__284" "T_COL__284_782","A4"."T_COL__285"           
"T_COL__285_783","A4"."T_COL__286" "T_COL__286_784","A4"."T_COL__287"           
"T_COL__287_785","A4"."T_COL__288" "T_COL__288_786","A4"."T_COL__289"           
"T_COL__289_787","A4"."T_COL__290" "T_COL__290_788","A4"."T_COL__291"           
"T_COL__291_789","A4"."T_COL__292" "T_COL__292_790","A4"."T_COL__293"           
"T_COL__293_791","A4"."T_COL__294" "T_COL__294_792","A4"."T_COL__295"           
"T_COL__295_793","A4"."T_COL__296" "T_COL__296_794","A4"."T_COL__297"           
"T_COL__297_795","A4"."T_COL__298" "T_COL__298_796","A4"."T_COL__299"           
"T_COL__299_797","A4"."T_COL__300" "T_COL__300_798","A4"."T_COL__301"           
"T_COL__301_799","A4"."T_COL__302" "T_COL__302_800","A4"."T_COL__303"           
"T_COL__303_801","A4"."T_COL__304" "T_COL__304_802","A4"."T_COL__305"           
"T_COL__305_803","A4"."T_COL__306" "T_COL__306_804","A4"."T_COL__307"           
"T_COL__307_805","A4"."T_COL__308" "T_COL__308_806","A4"."T_COL__309"           
"T_COL__309_807","A4"."T_COL__310" "T_COL__310_808","A4"."T_COL__311"           
"T_COL__311_809","A4"."T_COL__312" "T_COL__312_810","A4"."T_COL__313"           
"T_COL__313_811","A4"."T_COL__314" "T_COL__314_812","A4"."T_COL__315"           
"T_COL__315_813","A4"."T_COL__316" "T_COL__316_814","A4"."T_COL__317"           
"T_COL__317_815","A4"."T_COL__318" "T_COL__318_816","A4"."T_COL__319"           
"T_COL__319_817","A4"."T_COL__320" "T_COL__320_818","A4"."T_COL__321"           
"T_COL__321_819","A4"."T_COL__322" "T_COL__322_820","A4"."T_COL__323"           
"T_COL__323_821","A4"."T_COL__324" "T_COL__324_822","A4"."T_COL__325"           
"T_COL__325_823","A4"."T_COL__326" "T_COL__326_824","A4"."T_COL__327"           
"T_COL__327_825","A4"."T_COL__328" "T_COL__328_826","A4"."T_COL__329"           
"T_COL__329_827","A4"."T_COL__330" "T_COL__330_828","A4"."T_COL__331"           
"T_COL__331_829","A4"."T_COL__332" "T_COL__332_830","A4"."T_COL__333"           
"T_COL__333_831","A4"."T_COL__334" "T_COL__334_832","A4"."T_COL__335"           
"T_COL__335_833","A4"."T_COL__336" "T_COL__336_834","A4"."T_COL__337"           
"T_COL__337_835","A4"."T_COL__338" "T_COL__338_836","A4"."T_COL__339"           
"T_COL__339_837","A4"."T_COL__340" "T_COL__340_838","A4"."T_COL__341"           
"T_COL__341_839","A4"."T_COL__342" "T_COL__342_840","A4"."T_COL__343"           
"T_COL__343_841","A4"."T_COL__344" "T_COL__344_842","A4"."T_COL__345"           
"T_COL__345_843","A4"."T_COL__346" "T_COL__346_844","A4"."T_COL__347"           
"T_COL__347_845","A4"."T_COL__348" "T_COL__348_846","A4"."T_COL__349"           
"T_COL__349_847","A4"."T_COL__350" "T_COL__350_848","A4"."T_COL__351"           
"T_COL__351_849","A4"."T_COL__352" "T_COL__352_850","A4"."T_COL__353"           
"T_COL__353_851","A4"."T_COL__354" "T_COL__354_852","A4"."T_COL__355"           
"T_COL__355_853","A4"."T_COL__356" "T_COL__356_854","A4"."T_COL__357"           
"T_COL__357_855","A4"."T_COL__358" "T_COL__358_856","A4"."T_COL__359"           
"T_COL__359_857","A4"."T_COL__360" "T_COL__360_858","A4"."T_COL__361"           
"T_COL__361_859","A4"."T_COL__362" "T_COL__362_860","A4"."T_COL__363"           
"T_COL__363_861","A4"."T_COL__364" "T_COL__364_862","A4"."T_COL__365"           
"T_COL__365_863","A4"."T_COL__366" "T_COL__366_864","A4"."T_COL__367"           
"T_COL__367_865","A4"."T_COL__368" "T_COL__368_866","A4"."T_COL__369"           
"T_COL__369_867","A4"."T_COL__370" "T_COL__370_868","A4"."T_COL__371"           
"T_COL__371_869","A4"."T_COL__372" "T_COL__372_870","A4"."T_COL__373"           
"T_COL__373_871","A4"."T_COL__374" "T_COL__374_872","A4"."T_COL__375"           
"T_COL__375_873","A4"."T_COL__376" "T_COL__376_874","A4"."T_COL__377"           
"T_COL__377_875","A4"."T_COL__378" "T_COL__378_876","A4"."T_COL__379"           
"T_COL__379_877","A4"."T_COL__380" "T_COL__380_878","A4"."T_COL__381"           
"T_COL__381_879","A4"."T_COL__382" "T_COL__382_880","A4"."T_COL__383"           
"T_COL__383_881","A4"."T_COL__384" "T_COL__384_882","A4"."T_COL__385"           
"T_COL__385_883","A4"."T_COL__386" "T_COL__386_884","A4"."T_COL__387"           
"T_COL__387_885","A4"."T_COL__388" "T_COL__388_886","A4"."T_COL__389"           
"T_COL__389_887","A4"."T_COL__390" "T_COL__390_888","A4"."T_COL__391"           
"T_COL__391_889","A4"."T_COL__392" "T_COL__392_890","A4"."T_COL__393"           
"T_COL__393_891","A4"."T_COL__394" "T_COL__394_892","A4"."T_COL__395"           
"T_COL__395_893","A4"."T_COL__396" "T_COL__396_894","A4"."T_COL__397"           
"T_COL__397_895","A4"."T_COL__398" "T_COL__398_896","A4"."T_COL__399"           
"T_COL__399_897","A4"."T_COL__400" "T_COL__400_898","A4"."T_COL__401"           
"T_COL__401_899","A4"."T_COL__402" "T_COL__402_900","A4"."T_COL__403"           
"T_COL__403_901","A4"."T_COL__404" "T_COL__404_902","A4"."T_COL__405"           
"T_COL__405_903","A4"."T_COL__406" "T_COL__406_904","A4"."T_COL__407"           
"T_COL__407_905","A4"."T_COL__408" "T_COL__408_906","A4"."T_COL__409"           
"T_COL__409_907","A4"."T_COL__410" "T_COL__410_908","A4"."T_COL__411"           
"T_COL__411_909","A4"."T_COL__412" "T_COL__412_910","A4"."T_COL__413"           
"T_COL__413_911","A4"."T_COL__414" "T_COL__414_912","A4"."T_COL__415"           
"T_COL__415_913","A4"."T_COL__416" "T_COL__416_914","A4"."T_COL__417"           
"T_COL__417_915","A4"."T_COL__418" "T_COL__418_916","A4"."T_COL__419"           
"T_COL__419_917","A4"."T_COL__420" "T_COL__420_918","A4"."T_COL__421"           
"T_COL__421_919","A4"."T_COL__422" "T_COL__422_920","A4"."T_COL__423"           
"T_COL__423_921","A4"."T_COL__424" "T_COL__424_922","A4"."T_COL__425"           
"T_COL__425_923","A4"."T_COL__426" "T_COL__426_924","A4"."T_COL__427"           
"T_COL__427_925","A4"."T_COL__428" "T_COL__428_926","A4"."T_COL__429"           
"T_COL__429_927","A4"."T_COL__430" "T_COL__430_928","A4"."T_COL__431"           
"T_COL__431_929","A4"."T_COL__432" "T_COL__432_930","A4"."T_COL__433"           
"T_COL__433_931","A4"."T_COL__434" "T_COL__434_932","A4"."T_COL__435"           
"T_COL__435_933","A4"."T_COL__436" "T_COL__436_934","A4"."T_COL__437"           
"T_COL__437_935","A4"."T_COL__438" "T_COL__438_936","A4"."T_COL__439"           
"T_COL__439_937","A4"."T_COL__440" "T_COL__440_938","A4"."T_COL__441"           
"T_COL__441_939","A4"."T_COL__442" "T_COL__442_940","A4"."T_COL__443"           
"T_COL__443_941","A4"."T_COL__444" "T_COL__444_942","A4"."T_COL__445"           
"T_COL__445_943","A4"."T_COL__446" "T_COL__446_944","A4"."T_COL__447"           
"T_COL__447_945","A4"."T_COL__448" "T_COL__448_946","A4"."T_COL__449"           
"T_COL__449_947","A4"."T_COL__450" "T_COL__450_948","A4"."T_COL__451"           
"T_COL__451_949","A4"."T_COL__452" "T_COL__452_950","A4"."T_COL__453"           
"T_COL__453_951","A4"."T_COL__454" "T_COL__454_952","A4"."T_COL__455"           
"T_COL__455_953","A4"."T_COL__456" "T_COL__456_954","A4"."T_COL__457"           
"T_COL__457_955","A4"."T_COL__458" "T_COL__458_956","A4"."T_COL__459"           
"T_COL__459_957","A4"."T_COL__460" "T_COL__460_958","A4"."T_COL__461"           
"T_COL__461_959","A4"."T_COL__462" "T_COL__462_960","A4"."T_COL__463"           
"T_COL__463_961","A4"."T_COL__464" "T_COL__464_962","A4"."T_COL__465"           
"T_COL__465_963","A4"."T_COL__466" "T_COL__466_964","A4"."T_COL__467"           
"T_COL__467_965","A4"."T_COL__468" "T_COL__468_966","A4"."T_COL__469"           
"T_COL__469_967","A4"."T_COL__470" "T_COL__470_968","A4"."T_COL__471"           
"T_COL__471_969","A4"."T_COL__472" "T_COL__472_970","A4"."T_COL__473"           
"T_COL__473_971","A4"."T_COL__474" "T_COL__474_972","A4"."T_COL__475"           
"T_COL__475_973","A4"."T_COL__476" "T_COL__476_974","A4"."T_COL__477"           
"T_COL__477_975","A4"."T_COL__478" "T_COL__478_976","A4"."T_COL__479"           
"T_COL__479_977","A4"."T_COL__480" "T_COL__480_978","A4"."T_COL__481"           
"T_COL__481_979","A4"."T_COL__482" "T_COL__482_980","A4"."T_COL__483"           
"T_COL__483_981","A4"."T_COL__484" "T_COL__484_982","A4"."T_COL__485"           
"T_COL__485_983","A4"."T_COL__486" "T_COL__486_984","A4"."T_COL__487"           
"T_COL__487_985","A4"."T_COL__488" "T_COL__488_986","A4"."T_COL__489"           
"T_COL__489_987","A4"."T_COL__490" "T_COL__490_988","A4"."T_COL__491"           
"T_COL__491_989","A4"."T_COL__492" "T_COL__492_990","A4"."T_COL__493"           
"T_COL__493_991","A4"."T_COL__494" "T_COL__494_992","A4"."T_COL__495"           
"T_COL__495_993","A4"."T_COL__496" "T_COL__496_994","A4"."T_COL__497"           
"T_COL__497_995","A4"."T_COL__498" "T_COL__498_996","A4"."T_COL__499"           
"T_COL__499_997" FROM "SYS"."ORANGE" "A5","SYS"."MAUVE" "A4" WHERE              
"A5"."Z_COL_001"="A4"."Z_COL_001") "A3","SYS"."YELLOW" "A2" WHERE               
"A3"."Z_COL_001_0"="A2"."Z_COL_001") "A1"                                       
                                                                                

SQL> 
SQL> 
SQL> --********************************************************************************--
SQL> -- sub selecting only one column also fails in Oracle 12c:
SQL> -- SQL Error: ORA-01792: maximum number of columns in a table or view is 1000
SQL> --********************************************************************************--
SQL> 
SQL> select (select 1 from dual where 1 = 2) as exp_column
  2  from ORANGE g
  3    join MAUVE r on g.Z_COL_001 = r.Z_COL_001
  4    join YELLOW p on g.Z_COL_001 = p.Z_COL_001;

no rows selected

SQL> 
SQL> --********************************************************************************--
SQL> -- The same occurs here
SQL> -- Because of the errant expansion the total number of columns expands to just under
SQL> -- the query limit so that even selecting 1 column from causes the error to occur
SQL> --********************************************************************************--
SQL> set serveroutput on
SQL> set long 20000
SQL> 
SQL> variable m_sql_out clob
SQL> 
SQL> declare
  2  	 m_sql_in    clob :=
  3  			     '
  4  			     select (select 1 from dual where 1 = 2) as exp_column
  5  			     from ORANGE g
  6  			     join MAUVE r on g.Z_COL_001 = r.Z_COL_001
  7  			     join YELLOW p on g.Z_COL_001 = p.Z_COL_001
  8  			     ';
  9  begin
 10  
 11  	 dbms_utility.expand_sql_text(
 12  	     m_sql_in,
 13  	     :m_sql_out
 14  	 );
 15  
 16  end;
 17  /

PL/SQL procedure successfully completed.

SQL> 
SQL> column m_sql_out wrap word
SQL> print m_sql_out
SELECT  (SELECT 1 "1" FROM "SYS"."DUAL" "A2" WHERE 1=2) "EXP_COLUMN" FROM       
(SELECT "A4"."Z_COL_001_0" "Z_COL_001","A4"."Z_COL_002_1"                       
"Z_COL_002","A4"."Z_COL_003_2" "Z_COL_003","A4"."Z_COL_004_3"                   
"Z_COL_004","A4"."Z_COL_005_4" "Z_COL_005","A4"."Z_COL_006_5"                   
"Z_COL_006","A4"."Z_COL_007_6" "Z_COL_007","A4"."Z_COL_008_7"                   
"Z_COL_008","A4"."Z_COL_009_8" "Z_COL_009","A4"."Z_COL_010_9"                   
"Z_COL_010","A4"."Z_COL_011_10" "Z_COL_011","A4"."Z_COL_012_11"                 
"Z_COL_012","A4"."Z_COL_013_12" "Z_COL_013","A4"."Z_COL_014_13"                 
"Z_COL_014","A4"."Z_COL_015_14" "Z_COL_015","A4"."Z_COL_016_15"                 
"Z_COL_016","A4"."Z_COL_017_16" "Z_COL_017","A4"."Z_COL_018_17"                 
"Z_COL_018","A4"."Z_COL_019_18" "Z_COL_019","A4"."Z_COL_020_19"                 
"Z_COL_020","A4"."Z_COL_021_20" "Z_COL_021","A4"."Z_COL_022_21"                 
"Z_COL_022","A4"."Z_COL_023_22" "Z_COL_023","A4"."Z_COL_024_23"                 
"Z_COL_024","A4"."Z_COL_025_24" "Z_COL_025","A4"."Z_COL_026_25"                 
"Z_COL_026","A4"."Z_COL_027_26" "Z_COL_027","A4"."Z_COL_028_27"                 
"Z_COL_028","A4"."Z_COL_029_28" "Z_COL_029","A4"."Z_COL_030_29"                 
"Z_COL_030","A4"."Z_COL_031_30" "Z_COL_031","A4"."Z_COL_032_31"                 
"Z_COL_032","A4"."Z_COL_033_32" "Z_COL_033","A4"."Z_COL_034_33"                 
"Z_COL_034","A4"."Z_COL_035_34" "Z_COL_035","A4"."Z_COL_036_35"                 
"Z_COL_036","A4"."Z_COL_037_36" "Z_COL_037","A4"."Z_COL_038_37"                 
"Z_COL_038","A4"."Z_COL_039_38" "Z_COL_039","A4"."Z_COL_040_39"                 
"Z_COL_040","A4"."Z_COL_041_40" "Z_COL_041","A4"."Z_COL_042_41"                 
"Z_COL_042","A4"."Z_COL_043_42" "Z_COL_043","A4"."Z_COL_044_43"                 
"Z_COL_044","A4"."Z_COL_045_44" "Z_COL_045","A4"."Z_COL_046_45"                 
"Z_COL_046","A4"."Z_COL_047_46" "Z_COL_047","A4"."Z_COL_048_47"                 
"Z_COL_048","A4"."Z_COL_049_48" "Z_COL_049","A4"."Z_COL_050_49"                 
"Z_COL_050","A4"."Z_COL_051_50" "Z_COL_051","A4"."Z_COL_052_51"                 
"Z_COL_052","A4"."Z_COL_053_52" "Z_COL_053","A4"."Z_COL_054_53"                 
"Z_COL_054","A4"."Z_COL_055_54" "Z_COL_055","A4"."Z_COL_056_55"                 
"Z_COL_056","A4"."Z_COL_057_56" "Z_COL_057","A4"."Z_COL_058_57"                 
"Z_COL_058","A4"."Z_COL_059_58" "Z_COL_059","A4"."Z_COL_060_59"                 
"Z_COL_060","A4"."Z_COL_061_60" "Z_COL_061","A4"."Z_COL_062_61"                 
"Z_COL_062","A4"."Z_COL_063_62" "Z_COL_063","A4"."Z_COL_064_63"                 
"Z_COL_064","A4"."Z_COL_065_64" "Z_COL_065","A4"."Z_COL_066_65"                 
"Z_COL_066","A4"."Z_COL_067_66" "Z_COL_067","A4"."Z_COL_068_67"                 
"Z_COL_068","A4"."Z_COL_069_68" "Z_COL_069","A4"."Z_COL_070_69"                 
"Z_COL_070","A4"."Z_COL_071_70" "Z_COL_071","A4"."Z_COL_072_71"                 
"Z_COL_072","A4"."Z_COL_073_72" "Z_COL_073","A4"."Z_COL_074_73"                 
"Z_COL_074","A4"."Z_COL_075_74" "Z_COL_075","A4"."Z_COL_076_75"                 
"Z_COL_076","A4"."Z_COL_077_76" "Z_COL_077","A4"."Z_COL_078_77"                 
"Z_COL_078","A4"."Z_COL_079_78" "Z_COL_079","A4"."Z_COL_080_79"                 
"Z_COL_080","A4"."Z_COL_081_80" "Z_COL_081","A4"."Z_COL_082_81"                 
"Z_COL_082","A4"."Z_COL_083_82" "Z_COL_083","A4"."Z_COL_084_83"                 
"Z_COL_084","A4"."Z_COL_085_84" "Z_COL_085","A4"."Z_COL_086_85"                 
"Z_COL_086","A4"."Z_COL_087_86" "Z_COL_087","A4"."Z_COL_088_87"                 
"Z_COL_088","A4"."Z_COL_089_88" "Z_COL_089","A4"."Z_COL_090_89"                 
"Z_COL_090","A4"."Z_COL_091_90" "Z_COL_091","A4"."Z_COL_092_91"                 
"Z_COL_092","A4"."Z_COL_093_92" "Z_COL_093","A4"."Z_COL_094_93"                 
"Z_COL_094","A4"."Z_COL_095_94" "Z_COL_095","A4"."Z_COL_096_95"                 
"Z_COL_096","A4"."Z_COL_097_96" "Z_COL_097","A4"."Z_COL_098_97"                 
"Z_COL_098","A4"."Z_COL_099_98" "Z_COL_099","A4"."Z_COL_100_99"                 
"Z_COL_100","A4"."Z_COL_101_100" "Z_COL_101","A4"."Z_COL_102_101"               
"Z_COL_102","A4"."Z_COL_103_102" "Z_COL_103","A4"."Z_COL_104_103"               
"Z_COL_104","A4"."Z_COL_105_104" "Z_COL_105","A4"."Z_COL_106_105"               
"Z_COL_106","A4"."Z_COL_107_106" "Z_COL_107","A4"."Z_COL_108_107"               
"Z_COL_108","A4"."Z_COL_109_108" "Z_COL_109","A4"."Z_COL_110_109"               
"Z_COL_110","A4"."Z_COL_111_110" "Z_COL_111","A4"."Z_COL_112_111"               
"Z_COL_112","A4"."Z_COL_113_112" "Z_COL_113","A4"."Z_COL_114_113"               
"Z_COL_114","A4"."Z_COL_115_114" "Z_COL_115","A4"."Z_COL_116_115"               
"Z_COL_116","A4"."Z_COL_117_116" "Z_COL_117","A4"."Z_COL_118_117"               
"Z_COL_118","A4"."Z_COL_119_118" "Z_COL_119","A4"."Z_COL_120_119"               
"Z_COL_120","A4"."Z_COL_121_120" "Z_COL_121","A4"."Z_COL_122_121"               
"Z_COL_122","A4"."Z_COL_123_122" "Z_COL_123","A4"."Z_COL_124_123"               
"Z_COL_124","A4"."Z_COL_125_124" "Z_COL_125","A4"."Z_COL_126_125"               
"Z_COL_126","A4"."Z_COL_127_126" "Z_COL_127","A4"."Z_COL_128_127"               
"Z_COL_128","A4"."Z_COL_129_128" "Z_COL_129","A4"."Z_COL_130_129"               
"Z_COL_130","A4"."Z_COL_131_130" "Z_COL_131","A4"."Z_COL_132_131"               
"Z_COL_132","A4"."Z_COL_133_132" "Z_COL_133","A4"."Z_COL_134_133"               
"Z_COL_134","A4"."Z_COL_135_134" "Z_COL_135","A4"."Z_COL_136_135"               
"Z_COL_136","A4"."Z_COL_137_136" "Z_COL_137","A4"."Z_COL_138_137"               
"Z_COL_138","A4"."Z_COL_139_138" "Z_COL_139","A4"."Z_COL_140_139"               
"Z_COL_140","A4"."Z_COL_141_140" "Z_COL_141","A4"."Z_COL_142_141"               
"Z_COL_142","A4"."Z_COL_143_142" "Z_COL_143","A4"."Z_COL_144_143"               
"Z_COL_144","A4"."Z_COL_145_144" "Z_COL_145","A4"."Z_COL_146_145"               
"Z_COL_146","A4"."Z_COL_147_146" "Z_COL_147","A4"."Z_COL_148_147"               
"Z_COL_148","A4"."Z_COL_149_148" "Z_COL_149","A4"."Z_COL_150_149"               
"Z_COL_150","A4"."Z_COL_151_150" "Z_COL_151","A4"."Z_COL_152_151"               
"Z_COL_152","A4"."Z_COL_153_152" "Z_COL_153","A4"."Z_COL_154_153"               
"Z_COL_154","A4"."Z_COL_155_154" "Z_COL_155","A4"."Z_COL_156_155"               
"Z_COL_156","A4"."Z_COL_157_156" "Z_COL_157","A4"."Z_COL_158_157"               
"Z_COL_158","A4"."Z_COL_159_158" "Z_COL_159","A4"."Z_COL_160_159"               
"Z_COL_160","A4"."Z_COL_161_160" "Z_COL_161","A4"."Z_COL_162_161"               
"Z_COL_162","A4"."Z_COL_163_162" "Z_COL_163","A4"."Z_COL_164_163"               
"Z_COL_164","A4"."Z_COL_165_164" "Z_COL_165","A4"."Z_COL_166_165"               
"Z_COL_166","A4"."Z_COL_167_166" "Z_COL_167","A4"."Z_COL_168_167"               
"Z_COL_168","A4"."Z_COL_169_168" "Z_COL_169","A4"."Z_COL_170_169"               
"Z_COL_170","A4"."Z_COL_171_170" "Z_COL_171","A4"."Z_COL_172_171"               
"Z_COL_172","A4"."Z_COL_173_172" "Z_COL_173","A4"."Z_COL_174_173"               
"Z_COL_174","A4"."Z_COL_175_174" "Z_COL_175","A4"."Z_COL_176_175"               
"Z_COL_176","A4"."Z_COL_177_176" "Z_COL_177","A4"."Z_COL_178_177"               
"Z_COL_178","A4"."Z_COL_179_178" "Z_COL_179","A4"."Z_COL_180_179"               
"Z_COL_180","A4"."Z_COL_181_180" "Z_COL_181","A4"."Z_COL_182_181"               
"Z_COL_182","A4"."Z_COL_183_182" "Z_COL_183","A4"."Z_COL_184_183"               
"Z_COL_184","A4"."Z_COL_185_184" "Z_COL_185","A4"."Z_COL_186_185"               
"Z_COL_186","A4"."Z_COL_187_186" "Z_COL_187","A4"."Z_COL_188_187"               
"Z_COL_188","A4"."Z_COL_189_188" "Z_COL_189","A4"."Z_COL_190_189"               
"Z_COL_190","A4"."Z_COL_191_190" "Z_COL_191","A4"."Z_COL_192_191"               
"Z_COL_192","A4"."Z_COL_193_192" "Z_COL_193","A4"."Z_COL_194_193"               
"Z_COL_194","A4"."Z_COL_195_194" "Z_COL_195","A4"."Z_COL_196_195"               
"Z_COL_196","A4"."Z_COL_197_196" "Z_COL_197","A4"."Z_COL_198_197"               
"Z_COL_198","A4"."Z_COL_199_198" "Z_COL_199","A4"."Z_COL_200_199"               
"Z_COL_200","A4"."Z_COL_201_200" "Z_COL_201","A4"."Z_COL_202_201"               
"Z_COL_202","A4"."Z_COL_203_202" "Z_COL_203","A4"."Z_COL_204_203"               
"Z_COL_204","A4"."Z_COL_205_204" "Z_COL_205","A4"."Z_COL_206_205"               
"Z_COL_206","A4"."Z_COL_207_206" "Z_COL_207","A4"."Z_COL_208_207"               
"Z_COL_208","A4"."Z_COL_209_208" "Z_COL_209","A4"."Z_COL_210_209"               
"Z_COL_210","A4"."Z_COL_211_210" "Z_COL_211","A4"."Z_COL_212_211"               
"Z_COL_212","A4"."Z_COL_213_212" "Z_COL_213","A4"."Z_COL_214_213"               
"Z_COL_214","A4"."Z_COL_215_214" "Z_COL_215","A4"."Z_COL_216_215"               
"Z_COL_216","A4"."Z_COL_217_216" "Z_COL_217","A4"."Z_COL_218_217"               
"Z_COL_218","A4"."Z_COL_219_218" "Z_COL_219","A4"."Z_COL_220_219"               
"Z_COL_220","A4"."Z_COL_221_220" "Z_COL_221","A4"."Z_COL_222_221"               
"Z_COL_222","A4"."Z_COL_223_222" "Z_COL_223","A4"."Z_COL_224_223"               
"Z_COL_224","A4"."Z_COL_225_224" "Z_COL_225","A4"."Z_COL_226_225"               
"Z_COL_226","A4"."Z_COL_227_226" "Z_COL_227","A4"."Z_COL_228_227"               
"Z_COL_228","A4"."Z_COL_229_228" "Z_COL_229","A4"."Z_COL_230_229"               
"Z_COL_230","A4"."Z_COL_231_230" "Z_COL_231","A4"."Z_COL_232_231"               
"Z_COL_232","A4"."Z_COL_233_232" "Z_COL_233","A4"."Z_COL_234_233"               
"Z_COL_234","A4"."Z_COL_235_234" "Z_COL_235","A4"."Z_COL_236_235"               
"Z_COL_236","A4"."Z_COL_237_236" "Z_COL_237","A4"."Z_COL_238_237"               
"Z_COL_238","A4"."Z_COL_239_238" "Z_COL_239","A4"."Z_COL_240_239"               
"Z_COL_240","A4"."Z_COL_241_240" "Z_COL_241","A4"."Z_COL_242_241"               
"Z_COL_242","A4"."Z_COL_243_242" "Z_COL_243","A4"."Z_COL_244_243"               
"Z_COL_244","A4"."Z_COL_245_244" "Z_COL_245","A4"."Z_COL_246_245"               
"Z_COL_246","A4"."Z_COL_247_246" "Z_COL_247","A4"."Z_COL_248_247"               
"Z_COL_248","A4"."Z_COL_249_248" "Z_COL_249","A4"."Z_COL_250_249"               
"Z_COL_250","A4"."Z_COL_251_250" "Z_COL_251","A4"."Z_COL_252_251"               
"Z_COL_252","A4"."Z_COL_253_252" "Z_COL_253","A4"."Z_COL_254_253"               
"Z_COL_254","A4"."Z_COL_255_254" "Z_COL_255","A4"."Z_COL_256_255"               
"Z_COL_256","A4"."Z_COL_257_256" "Z_COL_257","A4"."Z_COL_258_257"               
"Z_COL_258","A4"."Z_COL_259_258" "Z_COL_259","A4"."Z_COL_260_259"               
"Z_COL_260","A4"."Z_COL_261_260" "Z_COL_261","A4"."Z_COL_262_261"               
"Z_COL_262","A4"."Z_COL_263_262" "Z_COL_263","A4"."Z_COL_264_263"               
"Z_COL_264","A4"."Z_COL_265_264" "Z_COL_265","A4"."Z_COL_266_265"               
"Z_COL_266","A4"."Z_COL_267_266" "Z_COL_267","A4"."Z_COL_268_267"               
"Z_COL_268","A4"."Z_COL_269_268" "Z_COL_269","A4"."Z_COL_270_269"               
"Z_COL_270","A4"."Z_COL_271_270" "Z_COL_271","A4"."Z_COL_272_271"               
"Z_COL_272","A4"."Z_COL_273_272" "Z_COL_273","A4"."Z_COL_274_273"               
"Z_COL_274","A4"."Z_COL_275_274" "Z_COL_275","A4"."Z_COL_276_275"               
"Z_COL_276","A4"."Z_COL_277_276" "Z_COL_277","A4"."Z_COL_278_277"               
"Z_COL_278","A4"."Z_COL_279_278" "Z_COL_279","A4"."Z_COL_280_279"               
"Z_COL_280","A4"."Z_COL_281_280" "Z_COL_281","A4"."Z_COL_282_281"               
"Z_COL_282","A4"."Z_COL_283_282" "Z_COL_283","A4"."Z_COL_284_283"               
"Z_COL_284","A4"."Z_COL_285_284" "Z_COL_285","A4"."Z_COL_286_285"               
"Z_COL_286","A4"."Z_COL_287_286" "Z_COL_287","A4"."Z_COL_288_287"               
"Z_COL_288","A4"."Z_COL_289_288" "Z_COL_289","A4"."Z_COL_290_289"               
"Z_COL_290","A4"."Z_COL_291_290" "Z_COL_291","A4"."Z_COL_292_291"               
"Z_COL_292","A4"."Z_COL_293_292" "Z_COL_293","A4"."Z_COL_294_293"               
"Z_COL_294","A4"."Z_COL_295_294" "Z_COL_295","A4"."Z_COL_296_295"               
"Z_COL_296","A4"."Z_COL_297_296" "Z_COL_297","A4"."Z_COL_298_297"               
"Z_COL_298","A4"."Z_COL_299_298" "Z_COL_299","A4"."Z_COL_300_299"               
"Z_COL_300","A4"."Z_COL_301_300" "Z_COL_301","A4"."Z_COL_302_301"               
"Z_COL_302","A4"."Z_COL_303_302" "Z_COL_303","A4"."Z_COL_304_303"               
"Z_COL_304","A4"."Z_COL_305_304" "Z_COL_305","A4"."Z_COL_306_305"               
"Z_COL_306","A4"."Z_COL_307_306" "Z_COL_307","A4"."Z_COL_308_307"               
"Z_COL_308","A4"."Z_COL_309_308" "Z_COL_309","A4"."Z_COL_310_309"               
"Z_COL_310","A4"."Z_COL_311_310" "Z_COL_311","A4"."Z_COL_312_311"               
"Z_COL_312","A4"."Z_COL_313_312" "Z_COL_313","A4"."Z_COL_314_313"               
"Z_COL_314","A4"."Z_COL_315_314" "Z_COL_315","A4"."Z_COL_316_315"               
"Z_COL_316","A4"."Z_COL_317_316" "Z_COL_317","A4"."Z_COL_318_317"               
"Z_COL_318","A4"."Z_COL_319_318" "Z_COL_319","A4"."Z_COL_320_319"               
"Z_COL_320","A4"."Z_COL_321_320" "Z_COL_321","A4"."Z_COL_322_321"               
"Z_COL_322","A4"."Z_COL_323_322" "Z_COL_323","A4"."Z_COL_324_323"               
"Z_COL_324","A4"."Z_COL_325_324" "Z_COL_325","A4"."Z_COL_326_325"               
"Z_COL_326","A4"."Z_COL_327_326" "Z_COL_327","A4"."Z_COL_328_327"               
"Z_COL_328","A4"."Z_COL_329_328" "Z_COL_329","A4"."Z_COL_330_329"               
"Z_COL_330","A4"."Z_COL_331_330" "Z_COL_331","A4"."Z_COL_332_331"               
"Z_COL_332","A4"."Z_COL_333_332" "Z_COL_333","A4"."Z_COL_334_333"               
"Z_COL_334","A4"."Z_COL_335_334" "Z_COL_335","A4"."Z_COL_336_335"               
"Z_COL_336","A4"."Z_COL_337_336" "Z_COL_337","A4"."Z_COL_338_337"               
"Z_COL_338","A4"."Z_COL_339_338" "Z_COL_339","A4"."Z_COL_340_339"               
"Z_COL_340","A4"."Z_COL_341_340" "Z_COL_341","A4"."Z_COL_342_341"               
"Z_COL_342","A4"."Z_COL_343_342" "Z_COL_343","A4"."Z_COL_344_343"               
"Z_COL_344","A4"."Z_COL_345_344" "Z_COL_345","A4"."Z_COL_346_345"               
"Z_COL_346","A4"."Z_COL_347_346" "Z_COL_347","A4"."Z_COL_348_347"               
"Z_COL_348","A4"."Z_COL_349_348" "Z_COL_349","A4"."Z_COL_350_349"               
"Z_COL_350","A4"."Z_COL_351_350" "Z_COL_351","A4"."Z_COL_352_351"               
"Z_COL_352","A4"."Z_COL_353_352" "Z_COL_353","A4"."Z_COL_354_353"               
"Z_COL_354","A4"."Z_COL_355_354" "Z_COL_355","A4"."Z_COL_356_355"               
"Z_COL_356","A4"."Z_COL_357_356" "Z_COL_357","A4"."Z_COL_358_357"               
"Z_COL_358","A4"."Z_COL_359_358" "Z_COL_359","A4"."Z_COL_360_359"               
"Z_COL_360","A4"."Z_COL_361_360" "Z_COL_361","A4"."Z_COL_362_361"               
"Z_COL_362","A4"."Z_COL_363_362" "Z_COL_363","A4"."Z_COL_364_363"               
"Z_COL_364","A4"."Z_COL_365_364" "Z_COL_365","A4"."Z_COL_366_365"               
"Z_COL_366","A4"."Z_COL_367_366" "Z_COL_367","A4"."Z_COL_368_367"               
"Z_COL_368","A4"."Z_COL_369_368" "Z_COL_369","A4"."Z_COL_370_369"               
"Z_COL_370","A4"."Z_COL_371_370" "Z_COL_371","A4"."Z_COL_372_371"               
"Z_COL_372","A4"."Z_COL_373_372" "Z_COL_373","A4"."Z_COL_374_373"               
"Z_COL_374","A4"."Z_COL_375_374" "Z_COL_375","A4"."Z_COL_376_375"               
"Z_COL_376","A4"."Z_COL_377_376" "Z_COL_377","A4"."Z_COL_378_377"               
"Z_COL_378","A4"."Z_COL_379_378" "Z_COL_379","A4"."Z_COL_380_379"               
"Z_COL_380","A4"."Z_COL_381_380" "Z_COL_381","A4"."Z_COL_382_381"               
"Z_COL_382","A4"."Z_COL_383_382" "Z_COL_383","A4"."Z_COL_384_383"               
"Z_COL_384","A4"."Z_COL_385_384" "Z_COL_385","A4"."Z_COL_386_385"               
"Z_COL_386","A4"."Z_COL_387_386" "Z_COL_387","A4"."Z_COL_388_387"               
"Z_COL_388","A4"."Z_COL_389_388" "Z_COL_389","A4"."Z_COL_390_389"               
"Z_COL_390","A4"."Z_COL_391_390" "Z_COL_391","A4"."Z_COL_392_391"               
"Z_COL_392","A4"."Z_COL_393_392" "Z_COL_393","A4"."Z_COL_394_393"               
"Z_COL_394","A4"."Z_COL_395_394" "Z_COL_395","A4"."Z_COL_396_395"               
"Z_COL_396","A4"."Z_COL_397_396" "Z_COL_397","A4"."Z_COL_398_397"               
"Z_COL_398","A4"."Z_COL_399_398" "Z_COL_399","A4"."Z_COL_400_399"               
"Z_COL_400","A4"."Z_COL_401_400" "Z_COL_401","A4"."Z_COL_402_401"               
"Z_COL_402","A4"."Z_COL_403_402" "Z_COL_403","A4"."Z_COL_404_403"               
"Z_COL_404","A4"."Z_COL_405_404" "Z_COL_405","A4"."Z_COL_406_405"               
"Z_COL_406","A4"."Z_COL_407_406" "Z_COL_407","A4"."Z_COL_408_407"               
"Z_COL_408","A4"."Z_COL_409_408" "Z_COL_409","A4"."Z_COL_410_409"               
"Z_COL_410","A4"."Z_COL_411_410" "Z_COL_411","A4"."Z_COL_412_411"               
"Z_COL_412","A4"."Z_COL_413_412" "Z_COL_413","A4"."Z_COL_414_413"               
"Z_COL_414","A4"."Z_COL_415_414" "Z_COL_415","A4"."Z_COL_416_415"               
"Z_COL_416","A4"."Z_COL_417_416" "Z_COL_417","A4"."Z_COL_418_417"               
"Z_COL_418","A4"."Z_COL_419_418" "Z_COL_419","A4"."Z_COL_420_419"               
"Z_COL_420","A4"."Z_COL_421_420" "Z_COL_421","A4"."Z_COL_422_421"               
"Z_COL_422","A4"."Z_COL_423_422" "Z_COL_423","A4"."Z_COL_424_423"               
"Z_COL_424","A4"."Z_COL_425_424" "Z_COL_425","A4"."Z_COL_426_425"               
"Z_COL_426","A4"."Z_COL_427_426" "Z_COL_427","A4"."Z_COL_428_427"               
"Z_COL_428","A4"."Z_COL_429_428" "Z_COL_429","A4"."Z_COL_430_429"               
"Z_COL_430","A4"."Z_COL_431_430" "Z_COL_431","A4"."Z_COL_432_431"               
"Z_COL_432","A4"."Z_COL_433_432" "Z_COL_433","A4"."Z_COL_434_433"               
"Z_COL_434","A4"."Z_COL_435_434" "Z_COL_435","A4"."Z_COL_436_435"               
"Z_COL_436","A4"."Z_COL_437_436" "Z_COL_437","A4"."Z_COL_438_437"               
"Z_COL_438","A4"."Z_COL_439_438" "Z_COL_439","A4"."Z_COL_440_439"               
"Z_COL_440","A4"."Z_COL_441_440" "Z_COL_441","A4"."Z_COL_442_441"               
"Z_COL_442","A4"."Z_COL_443_442" "Z_COL_443","A4"."Z_COL_444_443"               
"Z_COL_444","A4"."Z_COL_445_444" "Z_COL_445","A4"."Z_COL_446_445"               
"Z_COL_446","A4"."Z_COL_447_446" "Z_COL_447","A4"."Z_COL_448_447"               
"Z_COL_448","A4"."Z_COL_449_448" "Z_COL_449","A4"."Z_COL_450_449"               
"Z_COL_450","A4"."Z_COL_451_450" "Z_COL_451","A4"."Z_COL_452_451"               
"Z_COL_452","A4"."Z_COL_453_452" "Z_COL_453","A4"."Z_COL_454_453"               
"Z_COL_454","A4"."Z_COL_455_454" "Z_COL_455","A4"."Z_COL_456_455"               
"Z_COL_456","A4"."Z_COL_457_456" "Z_COL_457","A4"."Z_COL_458_457"               
"Z_COL_458","A4"."Z_COL_459_458" "Z_COL_459","A4"."Z_COL_460_459"               
"Z_COL_460","A4"."Z_COL_461_460" "Z_COL_461","A4"."Z_COL_462_461"               
"Z_COL_462","A4"."Z_COL_463_462" "Z_COL_463","A4"."Z_COL_464_463"               
"Z_COL_464","A4"."Z_COL_465_464" "Z_COL_465","A4"."Z_COL_466_465"               
"Z_COL_466","A4"."Z_COL_467_466" "Z_COL_467","A4"."Z_COL_468_467"               
"Z_COL_468","A4"."Z_COL_469_468" "Z_COL_469","A4"."Z_COL_470_469"               
"Z_COL_470","A4"."Z_COL_471_470" "Z_COL_471","A4"."Z_COL_472_471"               
"Z_COL_472","A4"."Z_COL_473_472" "Z_COL_473","A4"."Z_COL_474_473"               
"Z_COL_474","A4"."Z_COL_475_474" "Z_COL_475","A4"."Z_COL_476_475"               
"Z_COL_476","A4"."Z_COL_477_476" "Z_COL_477","A4"."Z_COL_478_477"               
"Z_COL_478","A4"."Z_COL_479_478" "Z_COL_479","A4"."Z_COL_480_479"               
"Z_COL_480","A4"."Z_COL_481_480" "Z_COL_481","A4"."Z_COL_482_481"               
"Z_COL_482","A4"."Z_COL_483_482" "Z_COL_483","A4"."Z_COL_484_483"               
"Z_COL_484","A4"."Z_COL_485_484" "Z_COL_485","A4"."Z_COL_486_485"               
"Z_COL_486","A4"."Z_COL_487_486" "Z_COL_487","A4"."Z_COL_488_487"               
"Z_COL_488","A4"."Z_COL_489_488" "Z_COL_489","A4"."Z_COL_490_489"               
"Z_COL_490","A4"."Z_COL_491_490" "Z_COL_491","A4"."Z_COL_492_491"               
"Z_COL_492","A4"."Z_COL_493_492" "Z_COL_493","A4"."Z_COL_494_493"               
"Z_COL_494","A4"."Z_COL_495_494" "Z_COL_495","A4"."Z_COL_496_495"               
"Z_COL_496","A4"."Z_COL_497_496" "Z_COL_497","A4"."Z_COL_498_497"               
"Z_COL_498","A4"."Z_COL_499_498" "Z_COL_499","A4"."Z_COL_001_499"               
"Z_COL_001","A4"."T_COL__002_500" "T_COL__002","A4"."T_COL__003_501"            
"T_COL__003","A4"."T_COL__004_502" "T_COL__004","A4"."T_COL__005_503"           
"T_COL__005","A4"."T_COL__006_504" "T_COL__006","A4"."T_COL__007_505"           
"T_COL__007","A4"."T_COL__008_506" "T_COL__008","A4"."T_COL__009_507"           
"T_COL__009","A4"."T_COL__010_508" "T_COL__010","A4"."T_COL__011_509"           
"T_COL__011","A4"."T_COL__012_510" "T_COL__012","A4"."T_COL__013_511"           
"T_COL__013","A4"."T_COL__014_512" "T_COL__014","A4"."T_COL__015_513"           
"T_COL__015","A4"."T_COL__016_514" "T_COL__016","A4"."T_COL__017_515"           
"T_COL__017","A4"."T_COL__018_516" "T_COL__018","A4"."T_COL__019_517"           
"T_COL__019","A4"."T_COL__020_518" "T_COL__020","A4"."T_COL__021_519"           
"T_COL__021","A4"."T_COL__022_520" "T_COL__022","A4"."T_COL__023_521"           
"T_COL__023","A4"."T_COL__024_522" "T_COL__024","A4"."T_COL__025_523"           
"T_COL__025","A4"."T_COL__026_524" "T_COL__026","A4"."T_COL__027_525"           
"T_COL__027","A4"."T_COL__028_526" "T_COL__028","A4"."T_COL__029_527"           
"T_COL__029","A4"."T_COL__030_528" "T_COL__030","A4"."T_COL__031_529"           
"T_COL__031","A4"."T_COL__032_530" "T_COL__032","A4"."T_COL__033_531"           
"T_COL__033","A4"."T_COL__034_532" "T_COL__034","A4"."T_COL__035_533"           
"T_COL__035","A4"."T_COL__036_534" "T_COL__036","A4"."T_COL__037_535"           
"T_COL__037","A4"."T_COL__038_536" "T_COL__038","A4"."T_COL__039_537"           
"T_COL__039","A4"."T_COL__040_538" "T_COL__040","A4"."T_COL__041_539"           
"T_COL__041","A4"."T_COL__042_540" "T_COL__042","A4"."T_COL__043_541"           
"T_COL__043","A4"."T_COL__044_542" "T_COL__044","A4"."T_COL__045_543"           
"T_COL__045","A4"."T_COL__046_544" "T_COL__046","A4"."T_COL__047_545"           
"T_COL__047","A4"."T_COL__048_546" "T_COL__048","A4"."T_COL__049_547"           
"T_COL__049","A4"."T_COL__050_548" "T_COL__050","A4"."T_COL__051_549"           
"T_COL__051","A4"."T_COL__052_550" "T_COL__052","A4"."T_COL__053_551"           
"T_COL__053","A4"."T_COL__054_552" "T_COL__054","A4"."T_COL__055_553"           
"T_COL__055","A4"."T_COL__056_554" "T_COL__056","A4"."T_COL__057_555"           
"T_COL__057","A4"."T_COL__058_556" "T_COL__058","A4"."T_COL__059_557"           
"T_COL__059","A4"."T_COL__060_558" "T_COL__060","A4"."T_COL__061_559"           
"T_COL__061","A4"."T_COL__062_560" "T_COL__062","A4"."T_COL__063_561"           
"T_COL__063","A4"."T_COL__064_562" "T_COL__064","A4"."T_COL__065_563"           
"T_COL__065","A4"."T_COL__066_564" "T_COL__066","A4"."T_COL__067_565"           
"T_COL__067","A4"."T_COL__068_566" "T_COL__068","A4"."T_COL__069_567"           
"T_COL__069","A4"."T_COL__070_568" "T_COL__070","A4"."T_COL__071_569"           
"T_COL__071","A4"."T_COL__072_570" "T_COL__072","A4"."T_COL__073_571"           
"T_COL__073","A4"."T_COL__074_572" "T_COL__074","A4"."T_COL__075_573"           
"T_COL__075","A4"."T_COL__076_574" "T_COL__076","A4"."T_COL__077_575"           
"T_COL__077","A4"."T_COL__078_576" "T_COL__078","A4"."T_COL__079_577"           
"T_COL__079","A4"."T_COL__080_578" "T_COL__080","A4"."T_COL__081_579"           
"T_COL__081","A4"."T_COL__082_580" "T_COL__082","A4"."T_COL__083_581"           
"T_COL__083","A4"."T_COL__084_582" "T_COL__084","A4"."T_COL__085_583"           
"T_COL__085","A4"."T_COL__086_584" "T_COL__086","A4"."T_COL__087_585"           
"T_COL__087","A4"."T_COL__088_586" "T_COL__088","A4"."T_COL__089_587"           
"T_COL__089","A4"."T_COL__090_588" "T_COL__090","A4"."T_COL__091_589"           
"T_COL__091","A4"."T_COL__092_590" "T_COL__092","A4"."T_COL__093_591"           
"T_COL__093","A4"."T_COL__094_592" "T_COL__094","A4"."T_COL__095_593"           
"T_COL__095","A4"."T_COL__096_594" "T_COL__096","A4"."T_COL__097_595"           
"T_COL__097","A4"."T_COL__098_596" "T_COL__098","A4"."T_COL__099_597"           
"T_COL__099","A4"."T_COL__100_598" "T_COL__100","A4"."T_COL__101_599"           
"T_COL__101","A4"."T_COL__102_600" "T_CO                                        
                                                                                

SQL> 
SQL> 
SQL> --
SQL> -- Selecting only one column from the ANSI join doesn't produce the exaggerated
SQL> -- column list
SQL> --
SQL> 
SQL> set serveroutput on
SQL> set long 20000
SQL> 
SQL> variable m_sql_out clob
SQL> 
SQL> declare
  2  	 m_sql_in    clob :=
  3  			     '
  4  			     select p.P_COLUMN_002
  5  			     from ORANGE g
  6  			     join MAUVE r on g.Z_COL_001 = r.Z_COL_001
  7  			     join YELLOW p on g.Z_COL_001 = p.Z_COL_001
  8  			     ';
  9  begin
 10  
 11  	 dbms_utility.expand_sql_text(
 12  	     m_sql_in,
 13  	     :m_sql_out
 14  	 );
 15  
 16  end;
 17  /

PL/SQL procedure successfully completed.

SQL> 
SQL> column m_sql_out wrap word
SQL> print m_sql_out

M_SQL_OUT                                                                       
--------------------------------------------------------------------------------
SELECT "A1"."P_COLUMN_002_3" "P_COLUMN_002" FROM  (SELECT "A3"."Z_COL_001_0"    
"Z_COL_001","A3"."Z_COL_001_1" "Z_COL_001","A2"."Z_COL_001"                     
"Z_COL_001","A2"."P_COLUMN_002" "P_COLUMN_002_3" FROM  (SELECT "A5"."Z_COL_001" 
"Z_COL_001_0","A4"."Z_COL_001" "Z_COL_001_1" FROM "SYS"."ORANGE"                
"A5","SYS"."MAUVE" "A4" WHERE "A5"."Z_COL_001"="A4"."Z_COL_001")                
"A3","SYS"."YELLOW" "A2" WHERE "A3"."Z_COL_001_0"="A2"."Z_COL_001") "A1"        
                                                                                

SQL> 
SQL> 
SQL> 
SQL> declare
  2  	 m_sql_in    clob :=
  3  			     '
  4  			     select p.*
  5  			     from ORANGE g,
  6  				  MAUVE r,
  7  				  YELLOW p
  8  			     where g.Z_COL_001 = p.Z_COL_001
  9  			     and g.Z_COL_001 = r.Z_COL_001
 10  			     ';
 11  begin
 12  
 13  	 dbms_utility.expand_sql_text(
 14  	     m_sql_in,
 15  	     :m_sql_out
 16  	 );
 17  
 18  end;
 19  /

PL/SQL procedure successfully completed.

SQL> 
SQL> column m_sql_out wrap word
SQL> print m_sql_out

M_SQL_OUT                                                                       
--------------------------------------------------------------------------------
SELECT "A1"."Z_COL_001" "Z_COL_001","A1"."P_COLUMN_002" "P_COLUMN_002" FROM     
"SYS"."ORANGE" "A3","SYS"."MAUVE" "A2","SYS"."YELLOW" "A1" WHERE                
"A3"."Z_COL_001"="A1"."Z_COL_001" AND "A3"."Z_COL_001"="A2"."Z_COL_001"         
                                                                                

SQL> 
SQL> 
SQL> drop table ORANGE;

Table dropped.

SQL> 
SQL> 
SQL> drop table MAUVE;

Table dropped.

SQL> 
SQL> 
SQL> drop table YELLOW;

Table dropped.

SQL> 

The workaround only prevents Oracle from throwing the ORA-01792 error, it doesn’t correct the inflated ANSI expansion nor is it supposed to. For now this is all that Oracle support has to offer on the issue. Hopefully this will be investigated and the underlying problem addressed. For now just know that this error can be thrown when the conditions reported here are met, and that there are two ways around it — re-write the query without ANSI joins or turn off the fix for bug 17376322. The first solution eliminates the base problem entirely; the second just masks the issue by refusing to throw the ORA-01792 error. It may be easier in the short run to use the _fix_control workaround until code can be rewritten; since it’s not clear if the underlying ANSI ezpansion problem can affect other aspects of query performance it may be best to ‘bite the bullet’ and re-write ANSI joins that use star expansion or subqueries into ‘old-style’ Oracle joins. But, that’s a decision only you can make.

And that’s what counts.

March 30, 2015

Sound Advice

Filed under: Performance — dfitzjarrell @ 09:33

"There is much worth noticing that often escapes the eye."
- Norton Juster, The Phantom Tollbooth

Oracle has provided views to assist in managing the SGA and PGA settings when you’re not using Automatic Memory Management (AMM) and a view to manage memory_target when you are using AMM. These views were designed to be fairly easy to interpret but it seems that there has been confusion of late in how to actually use the recommendations Oracle is presenting. Let’s fix that.

Starting with 10gR2 Oracle provided both V$SGA_TARGET_ADVICE and V$PGA_TARGET_ADVICE to assist the DBA in sizing the SGA and PGA based on how the database was using memory. The definitions of these views (as of 11.2.0.4) are shown below:


SQL> desc v$sga_target_advice
 Name                                                                     Null?    Type
 ------------------------------------------------------------------------ -------- -------------------------------------------------
 SGA_SIZE                                                                          NUMBER
 SGA_SIZE_FACTOR                                                                   NUMBER
 ESTD_DB_TIME                                                                      NUMBER
 ESTD_DB_TIME_FACTOR                                                               NUMBER
 ESTD_PHYSICAL_READS                                                               NUMBER

SQL> desc v$pga_target_advice
 Name                                                                     Null?    Type
 ------------------------------------------------------------------------ -------- -------------------------------------------------
 PGA_TARGET_FOR_ESTIMATE                                                           NUMBER
 PGA_TARGET_FACTOR                                                                 NUMBER
 ADVICE_STATUS                                                                     VARCHAR2(3)
 BYTES_PROCESSED                                                                   NUMBER
 ESTD_TIME                                                                         NUMBER
 ESTD_EXTRA_BYTES_RW                                                               NUMBER
 ESTD_PGA_CACHE_HIT_PERCENTAGE                                                     NUMBER
 ESTD_OVERALLOC_COUNT                                                              NUMBER

SQL>

Nothing unusual or esoteric here, however it appears that interpreting the provided advice can be confusing. Looking first at the contents of V$SGA_TARGET_ADVICE let’s see what Oracle reports:


SQL> select * From v$sga_target_advice
  2  /

  SGA_SIZE SGA_SIZE_FACTOR ESTD_DB_TIME ESTD_DB_TIME_FACTOR ESTD_PHYSICAL_READS
---------- --------------- ------------ ------------------- -------------------
       520              .5        65793              5.8208             2579079
       780             .75        23911              2.1155             2068005
      1040               1        11303                   1             1134459
      1300            1.25         9162               .8106              970643
      1560             1.5         9098               .8049              970643
      1820            1.75         9058               .8014              970643
      2080               2         9058               .8014              970643

7 rows selected.

SQL>

Based on resource usage since startup this view provides information on SGA memory settings that may improve performance if adopted. The key element in any of these views is the size factor, with a size factor of 1 being the current setting. Looking at the above output it’s possible that physical reads can be reduced a small amount if sga_target is increased from 1040 MB to 1300 MB; any further increase to sga_target will show no improvement. This is proven by the ESTD_PHYSICAL_READS column, the value of which stays constant for an SGA sized at 1300 MB or greater. In addition the ESTD_DB_TIME_FACTOR doesn’t improve much at greater SGA sizes indicating that further increases provide no real benefit. Ideally the goal is to find the setting providing substantial benefit and use it; the problem appears to be ‘where do I stop looking?’ With the above output we stopped looking for improvement at the first entry providing the lowest estimated physical reads. All entries below that provided no additional benefit.

If sga_target and sga_max_size are set diffently (to allow some degree of dynamic SGA management) it may not be necessary to restart the database since sga_target may be able to be set without adjusting sga_max_size. This view may, however, provide an SGA setting that requires both sga_target and sga_max_size be adjusted, and in that case the database will need to be restarted.

In a similar fashion the V$PGA_TARGET_ADVICE view can also be used to ‘fine tune’ overall PGA settings for the database:


SQL> select * From v$pga_target_advice;

PGA_TARGET_FOR_ESTIMATE PGA_TARGET_FACTOR ADV BYTES_PROCESSED  ESTD_TIME ESTD_EXTRA_BYTES_RW ESTD_PGA_CACHE_HIT_PERCENTAGE ESTD_OVERALLOC_COUNT
----------------------- ----------------- --- --------------- ---------- ------------------- ----------------------------- --------------------
               73400320              .125 ON       7.4905E+10   37173880          1395450880                            98                   88
              146800640               .25 ON       7.4905E+10   36716537           456747008                            99                    6
              293601280                .5 ON       7.4905E+10   36643991           307845120                           100                    0
              440401920               .75 ON       7.4905E+10   36494007                   0                           100                    0
              587202560                 1 ON       7.4905E+10   36494007                   0                           100                    0
              704643072               1.2 ON       7.4905E+10   36494007                   0                           100                    0
              822083584               1.4 ON       7.4905E+10   36494007                   0                           100                    0
              939524096               1.6 ON       7.4905E+10   36494007                   0                           100                    0
             1056964608               1.8 ON       7.4905E+10   36494007                   0                           100                    0
             1174405120                 2 ON       7.4905E+10   36494007                   0                           100                    0
             1761607680                 3 ON       7.4905E+10   36494007                   0                           100                    0

PGA_TARGET_FOR_ESTIMATE PGA_TARGET_FACTOR ADV BYTES_PROCESSED  ESTD_TIME ESTD_EXTRA_BYTES_RW ESTD_PGA_CACHE_HIT_PERCENTAGE ESTD_OVERALLOC_COUNT
----------------------- ----------------- --- --------------- ---------- ------------------- ----------------------------- --------------------
             2348810240                 4 ON       7.4905E+10   36494007                   0                           100                    0
             3523215360                 6 ON       7.4905E+10   36494007                   0                           100                    0
             4697620480                 8 ON       7.4905E+10   36494007                   0                           100                    0

14 rows selected.

SQL>

Again it’s the target factor that identifies the current setting; working toward larger target factors may show improvement in the PGA cache hit percentage which means processes will see more optimized (one-pass) reads. Like the V$SGA_TARGET_ADVICE view the method is to find a setting that provides improvement. In this example the PGA setting is already optimal (the cache hit percentage is 100 and 0 extra bytes will need to be read). Not every database will be so lucky; this view is provided to allow the DBA to make as few changes as possible to achieve improved performance and throughput, eliminating the need for trial and error PGA management. In this case the database will need to be restarted after the parameter values have been changed.

To restate the methodology of using these two views:


1.	Find the row referencing the current setting; this will be the row where the size factor is equal to 1
2.	Scan the data for the entry just before the improvement estimates no longer change; this will be the setting
	Oracle has calculated to provide the most benefit
3.	Implement that setting, restarting the database, if necessary, to effect the change

In a similar fashion when using Automatic Memory Management, by setting memory_target and memory_max_target, the V$MEMORY_TARGET_ADVICE view can be used to tune the overall memory settings. Output from this view looks like this:


SQL> select * From v$memory_target_advice;

MEMORY_SIZE MEMORY_SIZE_FACTOR ESTD_DB_TIME ESTD_DB_TIME_FACTOR    VERSION     CON_ID
----------- ------------------ ------------ ------------------- ---------- ----------
        392                .25       143586              1.0063          0          0
        784                 .5       142687                   1          0          0
       1176                .75       142687                   1          0          0
       1568                  1       142687                   1          0          0
       1960               1.25       142687                   1          0          0
       2352                1.5       142687                   1          0          0
       2744               1.75       142687                   1          0          0
       3136                  2       142687                   1          0          0

8 rows selected.

SQL>

The same methodology is used here, and if memory_max_target is larger than the suggested memory_target value the setting can be changed without restarting the database. Like the SGA settings if memory_max_target needs to also be adjusted this will require a database restart. Note that if you’re using Linux and hugepages using AMM is not supported.

Managing database memory settings doesn’t need to be a gruelling task of ‘try this, see if it works, it doesn’t so try another setting and repeat’. Using the provided memory advisors can make the DBA’s job much easier. Hopefully this has made that task easier, freeing the DBA to address more pressing issues.

Sometimes it’s good to take things under advisement.

March 12, 2015

Extra-cost Option?

Filed under: General,Performance — dfitzjarrell @ 09:14

"Just because you have a choice, it doesn't mean that any of them 'has' to be right."
 Norton Juster, The Phantom Tollbooth 

In an Oracle forum that I frequent a user posted that he found cardinality values to be severely inflated when using an analytic function. This user posted an example that will be used here to demonstrate the problem. It will also be used to demonstrate another issue that can make this problem even more confusing, but the latter issue is one that can easily be fixed.

The example script contains the following code, which includes statements not executed by the user posting the problem (creating a new user to run the example from, using an additional statistics gathering method to verify the method wasn’t part of the cause and modifying the dynamic sampling setting), but which have no effect on the results generated:



connect bing/#############

spool wrong_card_ex.log
select * From v$version;

set echo on linesize 132

create table t1
as
select * From all_objects;

exec dbms_stats.gather_table_stats(user, 'T1')

EXPLAIN PLAN FOR
SELECT object_type
     , object_name
FROM (
	  SELECT object_type
	       , object_name
	  FROM   t1
)
WHERE object_name = 'DUAL';
SELECT * FROM TABLE(dbms_xplan.display);

set autotrace on

select object_type, object_name
from
(select object_type, object_name from t1)
where object_name = 'DUAL';

select object_type, object_name, object_rank
from
(select object_type, object_name, row_number() over (partition by object_type order by created) object_rank from t1)
where object_name = 'DUAL';

set autotrace off

connect bing/#############

set echo on linesize 132

alter session set optimizer_dynamic_sampling=11;

EXPLAIN PLAN FOR
SELECT object_type
     , object_name
     , object_rank
FROM (
	  SELECT object_type
	       , object_name
	       , ROW_NUMBER() OVER (PARTITION BY object_type ORDER BY created) object_rank
	  FROM   t1
)
WHERE object_name = 'DUAL';
SELECT * FROM TABLE(dbms_xplan.display);

set autotrace on

select object_type, object_name
from
(select object_type, object_name from t1)
where object_name = 'DUAL';

select object_type, object_name, object_rank
from
(select object_type, object_name, row_number() over (partition by object_type order by created) object_rank from t1)
where object_name = 'DUAL';

set autotrace off

exec dbms_stats.gather_schema_stats(user)

EXPLAIN PLAN FOR
SELECT object_type
     , object_name
FROM (
	  SELECT object_type
	       , object_name
	  FROM   t1
)
WHERE object_name = 'DUAL';
SELECT * FROM TABLE(dbms_xplan.display);

set autotrace on

select object_type, object_name
from
(select object_type, object_name from t1)
where object_name = 'DUAL';

select object_type, object_name, object_rank
from
(select object_type, object_name, row_number() over (partition by object_type order by created) object_rank from t1)
where object_name = 'DUAL';

set autotrace off

connect bing/#############

set echo on linesize 132

alter session set optimizer_dynamic_sampling=11;

EXPLAIN PLAN FOR
SELECT object_type
     , object_name
     , object_rank
FROM (
	  SELECT object_type
	       , object_name
	       , ROW_NUMBER() OVER (PARTITION BY object_type ORDER BY created) object_rank
	  FROM   t1
)
WHERE object_name = 'DUAL';
SELECT * FROM TABLE(dbms_xplan.display);

set autotrace on

select object_type, object_name
from
(select object_type, object_name from t1)
where object_name = 'DUAL';

select object_type, object_name, object_rank
from
(select object_type, object_name, row_number() over (partition by object_type order by created) object_rank from t1)
where object_name = 'DUAL';

set autotrace off

spool off

Nothing unusual there, really, other than the second run of the code minus the table creation. That was done to test whether gathering schema stats over gathering table stats made a difference in the cardinality numbers; it did not. Let’s look at a run of this script in an 11.2.0.4 database and see what cardinalities Oracle generates:


BANNER                                                                                                                              
--------------------------------------------------------------------------------                                                    
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production                                                        
PL/SQL Release 11.2.0.4.0 - Production                                                                                              
CORE	11.2.0.4.0	Production                                                                                                          
TNS for 64-bit Windows: Version 11.2.0.4.0 - Production                                                                             
NLSRTL Version 11.2.0.4.0 - Production                                                                                              

SQL> 
SQL> create table t1
  2  as
  3  select * From all_objects;

Table created.

SQL> 
SQL> exec dbms_stats.gather_table_stats(user, 'T1')

PL/SQL procedure successfully completed.

SQL> 
SQL> EXPLAIN PLAN FOR
  2  SELECT object_type
  3  	  , object_name
  4  FROM (
  5  	       SELECT object_type
  6  		    , object_name
  7  	       FROM   t1
  8  )
  9  WHERE object_name = 'DUAL';

Explained.

SQL> SELECT * FROM TABLE(dbms_xplan.display);

PLAN_TABLE_OUTPUT                                                                                                                   
------------------------------------------------------------------------------------------------------------------------------------
Plan hash value: 3617692013                                                                                                         
                                                                                                                                    
--------------------------------------------------------------------------                                                          
| Id  | Operation         | Name | Rows  | Bytes | Cost (%CPU)| Time     |                                                          
--------------------------------------------------------------------------                                                          
|   0 | SELECT STATEMENT  |      |     2 |    72 |   274   (1)| 00:00:04 |                                                          
|*  1 |  TABLE ACCESS FULL| T1   |     2 |    72 |   274   (1)| 00:00:04 |                                                          
--------------------------------------------------------------------------                                                          
                                                                                                                                    
Predicate Information (identified by operation id):                                                                                 
---------------------------------------------------                                                                                 

PLAN_TABLE_OUTPUT                                                                                                                   
------------------------------------------------------------------------------------------------------------------------------------
                                                                                                                                    
   1 - filter("OBJECT_NAME"='DUAL')                                                                                                 

13 rows selected.

SQL> 
SQL> set autotrace on
SQL> 
SQL> select object_type, object_name
  2  from
  3  (select object_type, object_name from t1)
  4  where object_name = 'DUAL';

OBJECT_TYPE         OBJECT_NAME                                                                                                     
------------------- ------------------------------                                                                                  
TABLE               DUAL                                                                                                            
SYNONYM             DUAL                                                                                                            


Execution Plan
----------------------------------------------------------                                                                          
Plan hash value: 3617692013                                                                                                         
                                                                                                                                    
--------------------------------------------------------------------------                                                          
| Id  | Operation         | Name | Rows  | Bytes | Cost (%CPU)| Time     |                                                          
--------------------------------------------------------------------------                                                          
|   0 | SELECT STATEMENT  |      |     2 |    72 |   274   (1)| 00:00:04 |                                                          
|*  1 |  TABLE ACCESS FULL| T1   |     2 |    72 |   274   (1)| 00:00:04 |                                                          
--------------------------------------------------------------------------                                                          
                                                                                                                                    
Predicate Information (identified by operation id):                                                                                 
---------------------------------------------------                                                                                 
                                                                                                                                    
   1 - filter("OBJECT_NAME"='DUAL')                                                                                                 


Statistics
----------------------------------------------------------                                                                          
          1  recursive calls                                                                                                        
          0  db block gets                                                                                                          
        983  consistent gets                                                                                                        
          0  physical reads                                                                                                         
          0  redo size                                                                                                              
        674  bytes sent via SQL*Net to client                                                                                       
        499  bytes received via SQL*Net from client                                                                                 
          2  SQL*Net roundtrips to/from client                                                                                      
          0  sorts (memory)                                                                                                         
          0  sorts (disk)                                                                                                           
          2  rows processed                                                                                                         

SQL> 
SQL> select object_type, object_name, object_rank
  2  from
  3  (select object_type, object_name, row_number() over (partition by object_type order by created) object_rank from t1)
  4  where object_name = 'DUAL';

OBJECT_TYPE         OBJECT_NAME                    OBJECT_RANK                                                                      
------------------- ------------------------------ -----------                                                                      
SYNONYM             DUAL                                     1                                                                      
TABLE               DUAL                                    43                                                                      


Execution Plan
----------------------------------------------------------                                                                          
Plan hash value: 2273146475                                                                                                         
                                                                                                                                    
------------------------------------------------------------------------------------                                                
| Id  | Operation           | Name | Rows  | Bytes |TempSpc| Cost (%CPU)| Time     |                                                
------------------------------------------------------------------------------------                                                
|   0 | SELECT STATEMENT    |      | 68395 |  2738K|       |  1042   (1)| 00:00:13 |                                                
|*  1 |  VIEW               |      | 68395 |  2738K|       |  1042   (1)| 00:00:13 |                                                
|   2 |   WINDOW SORT       |      | 68395 |  2938K|  3784K|  1042   (1)| 00:00:13 |                                                
|   3 |    TABLE ACCESS FULL| T1   | 68395 |  2938K|       |   274   (1)| 00:00:04 |                                                
------------------------------------------------------------------------------------                                                
                                                                                                                                    
Predicate Information (identified by operation id):                                                                                 
---------------------------------------------------                                                                                 
                                                                                                                                    
   1 - filter("OBJECT_NAME"='DUAL')                                                                                                 


Statistics
----------------------------------------------------------                                                                          
          1  recursive calls                                                                                                        
          0  db block gets                                                                                                          
        982  consistent gets                                                                                                        
          0  physical reads                                                                                                         
          0  redo size                                                                                                              
        753  bytes sent via SQL*Net to client                                                                                       
        499  bytes received via SQL*Net from client                                                                                 
          2  SQL*Net roundtrips to/from client                                                                                      
          1  sorts (memory)                                                                                                         
          0  sorts (disk)                                                                                                           
          2  rows processed                                                                                                         

SQL> 
SQL> set autotrace off
SQL> 
SQL> connect bing/##########
Connected.
SQL> 
SQL> set echo on linesize 132
SQL> 
SQL> alter session set optimizer_dynamic_sampling=11;

Session altered.

SQL> 
SQL> EXPLAIN PLAN FOR
  2  SELECT object_type
  3  	  , object_name
  4  	  , object_rank
  5  FROM (
  6  	       SELECT object_type
  7  		    , object_name
  8  		    , ROW_NUMBER() OVER (PARTITION BY object_type ORDER BY created) object_rank
  9  	       FROM   t1
 10  )
 11  WHERE object_name = 'DUAL';

Explained.

SQL> SELECT * FROM TABLE(dbms_xplan.display);

PLAN_TABLE_OUTPUT                                                                                                                   
------------------------------------------------------------------------------------------------------------------------------------
Plan hash value: 2273146475                                                                                                         
                                                                                                                                    
------------------------------------------------------------------------------------                                                
| Id  | Operation           | Name | Rows  | Bytes |TempSpc| Cost (%CPU)| Time     |                                                
------------------------------------------------------------------------------------                                                
|   0 | SELECT STATEMENT    |      | 68395 |  2738K|       |  1042   (1)| 00:00:13 |                                                
|*  1 |  VIEW               |      | 68395 |  2738K|       |  1042   (1)| 00:00:13 |                                                
|   2 |   WINDOW SORT       |      | 68395 |  2938K|  3784K|  1042   (1)| 00:00:13 |                                                
|   3 |    TABLE ACCESS FULL| T1   | 68395 |  2938K|       |   274   (1)| 00:00:04 |                                                
------------------------------------------------------------------------------------                                                
                                                                                                                                    

PLAN_TABLE_OUTPUT                                                                                                                   
------------------------------------------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):                                                                                 
---------------------------------------------------                                                                                 
                                                                                                                                    
   1 - filter("OBJECT_NAME"='DUAL')                                                                                                 
                                                                                                                                    
Note                                                                                                                                
-----                                                                                                                               
   - dynamic sampling used for this statement (level=11)                                                                            

19 rows selected.

SQL> 
SQL> set autotrace on
SQL> 
SQL> select object_type, object_name
  2  from
  3  (select object_type, object_name from t1)
  4  where object_name = 'DUAL';

OBJECT_TYPE         OBJECT_NAME                                                                                                     
------------------- ------------------------------                                                                                  
TABLE               DUAL                                                                                                            
SYNONYM             DUAL                                                                                                            


Execution Plan
----------------------------------------------------------                                                                          
Plan hash value: 3617692013                                                                                                         
                                                                                                                                    
--------------------------------------------------------------------------                                                          
| Id  | Operation         | Name | Rows  | Bytes | Cost (%CPU)| Time     |                                                          
--------------------------------------------------------------------------                                                          
|   0 | SELECT STATEMENT  |      |     2 |    72 |   274   (1)| 00:00:04 |                                                          
|*  1 |  TABLE ACCESS FULL| T1   |     2 |    72 |   274   (1)| 00:00:04 |                                                          
--------------------------------------------------------------------------                                                          
                                                                                                                                    
Predicate Information (identified by operation id):                                                                                 
---------------------------------------------------                                                                                 
                                                                                                                                    
   1 - filter("OBJECT_NAME"='DUAL')                                                                                                 
                                                                                                                                    
Note                                                                                                                                
-----                                                                                                                               
   - dynamic sampling used for this statement (level=11)                                                                            


Statistics
----------------------------------------------------------                                                                          
         12  recursive calls                                                                                                        
          0  db block gets                                                                                                          
       1788  consistent gets                                                                                                        
          1  physical reads                                                                                                         
          0  redo size                                                                                                              
        674  bytes sent via SQL*Net to client                                                                                       
        500  bytes received via SQL*Net from client                                                                                 
          2  SQL*Net roundtrips to/from client                                                                                      
          0  sorts (memory)                                                                                                         
          0  sorts (disk)                                                                                                           
          2  rows processed                                                                                                         

SQL> 

So far, so good, as the cardinality accurately reflects the actual number of rows for DUAL in the ALL_OBJECTS view. Let’s add the ROW_NUMBER() analytic function and see what changes:


SQL> select object_type, object_name, object_rank
  2  from
  3  (select object_type, object_name, row_number() over (partition by object_type order by created) object_rank from t1)
  4  where object_name = 'DUAL';

OBJECT_TYPE         OBJECT_NAME                    OBJECT_RANK                                                                      
------------------- ------------------------------ -----------                                                                      
SYNONYM             DUAL                                     1                                                                      
TABLE               DUAL                                    43                                                                      


Execution Plan
----------------------------------------------------------                                                                          
Plan hash value: 2273146475                                                                                                         
                                                                                                                                    
------------------------------------------------------------------------------------                                                
| Id  | Operation           | Name | Rows  | Bytes |TempSpc| Cost (%CPU)| Time     |                                                
------------------------------------------------------------------------------------                                                
|   0 | SELECT STATEMENT    |      | 68395 |  2738K|       |  1042   (1)| 00:00:13 |                                                
|*  1 |  VIEW               |      | 68395 |  2738K|       |  1042   (1)| 00:00:13 |                                                
|   2 |   WINDOW SORT       |      | 68395 |  2938K|  3784K|  1042   (1)| 00:00:13 |                                                
|   3 |    TABLE ACCESS FULL| T1   | 68395 |  2938K|       |   274   (1)| 00:00:04 |                                                
------------------------------------------------------------------------------------                                                
                                                                                                                                    
Predicate Information (identified by operation id):                                                                                 
---------------------------------------------------                                                                                 
                                                                                                                                    
   1 - filter("OBJECT_NAME"='DUAL')                                                                                                 
                                                                                                                                    
Note                                                                                                                                
-----                                                                                                                               
   - dynamic sampling used for this statement (level=11)                                                                            


Statistics
----------------------------------------------------------                                                                          
          7  recursive calls                                                                                                        
          0  db block gets                                                                                                          
        989  consistent gets                                                                                                        
          0  physical reads                                                                                                         
          0  redo size                                                                                                              
        753  bytes sent via SQL*Net to client                                                                                       
        500  bytes received via SQL*Net from client                                                                                 
          2  SQL*Net roundtrips to/from client                                                                                      
          1  sorts (memory)                                                                                                         
          0  sorts (disk)                                                                                                           
          2  rows processed                                                                                                         

SQL> 
SQL> set autotrace off

Interestingly (to put it mildly) the cardinality has jumped to over 30,000 times its original value. Maybe it’s the way the stats were gathered; another run is made, this time gathering stats for the entire schema (fortunately it contains only this one table):


SQL> 
SQL> exec dbms_stats.gather_schema_stats(user)

PL/SQL procedure successfully completed.

SQL> 
SQL> EXPLAIN PLAN FOR
  2  SELECT object_type
  3  	  , object_name
  4  FROM (
  5  	       SELECT object_type
  6  		    , object_name
  7  	       FROM   t1
  8  )
  9  WHERE object_name = 'DUAL';

Explained.

SQL> SELECT * FROM TABLE(dbms_xplan.display);

PLAN_TABLE_OUTPUT                                                                                                                   
------------------------------------------------------------------------------------------------------------------------------------
Plan hash value: 3617692013                                                                                                         
                                                                                                                                    
--------------------------------------------------------------------------                                                          
| Id  | Operation         | Name | Rows  | Bytes | Cost (%CPU)| Time     |                                                          
--------------------------------------------------------------------------                                                          
|   0 | SELECT STATEMENT  |      |     2 |    72 |   274   (1)| 00:00:04 |                                                          
|*  1 |  TABLE ACCESS FULL| T1   |     2 |    72 |   274   (1)| 00:00:04 |                                                          
--------------------------------------------------------------------------                                                          
                                                                                                                                    
Predicate Information (identified by operation id):                                                                                 
---------------------------------------------------                                                                                 

PLAN_TABLE_OUTPUT                                                                                                                   
------------------------------------------------------------------------------------------------------------------------------------
                                                                                                                                    
   1 - filter("OBJECT_NAME"='DUAL')                                                                                                 
                                                                                                                                    
Note                                                                                                                                
-----                                                                                                                               
   - dynamic sampling used for this statement (level=11)                                                                            

17 rows selected.

SQL> 
SQL> set autotrace on
SQL> 
SQL> select object_type, object_name
  2  from
  3  (select object_type, object_name from t1)
  4  where object_name = 'DUAL';

OBJECT_TYPE         OBJECT_NAME                                                                                                     
------------------- ------------------------------                                                                                  
TABLE               DUAL                                                                                                            
SYNONYM             DUAL                                                                                                            


Execution Plan
----------------------------------------------------------                                                                          
Plan hash value: 3617692013                                                                                                         
                                                                                                                                    
--------------------------------------------------------------------------                                                          
| Id  | Operation         | Name | Rows  | Bytes | Cost (%CPU)| Time     |                                                          
--------------------------------------------------------------------------                                                          
|   0 | SELECT STATEMENT  |      |     2 |    72 |   274   (1)| 00:00:04 |                                                          
|*  1 |  TABLE ACCESS FULL| T1   |     2 |    72 |   274   (1)| 00:00:04 |                                                          
--------------------------------------------------------------------------                                                          
                                                                                                                                    
Predicate Information (identified by operation id):                                                                                 
---------------------------------------------------                                                                                 
                                                                                                                                    
   1 - filter("OBJECT_NAME"='DUAL')                                                                                                 
                                                                                                                                    
Note                                                                                                                                
-----                                                                                                                               
   - dynamic sampling used for this statement (level=11)                                                                            


Statistics
----------------------------------------------------------                                                                          
          0  recursive calls                                                                                                        
          0  db block gets                                                                                                          
        983  consistent gets                                                                                                        
          0  physical reads                                                                                                         
          0  redo size                                                                                                              
        674  bytes sent via SQL*Net to client                                                                                       
        500  bytes received via SQL*Net from client                                                                                 
          2  SQL*Net roundtrips to/from client                                                                                      
          0  sorts (memory)                                                                                                         
          0  sorts (disk)                                                                                                           
          2  rows processed                                                                                                         

SQL> 
SQL> select object_type, object_name, object_rank
  2  from
  3  (select object_type, object_name, row_number() over (partition by object_type order by created) object_rank from t1)
  4  where object_name = 'DUAL';

OBJECT_TYPE         OBJECT_NAME                    OBJECT_RANK                                                                      
------------------- ------------------------------ -----------                                                                      
SYNONYM             DUAL                                     1                                                                      
TABLE               DUAL                                    43                                                                      


Execution Plan
----------------------------------------------------------                                                                          
Plan hash value: 2273146475                                                                                                         
                                                                                                                                    
------------------------------------------------------------------------------------                                                
| Id  | Operation           | Name | Rows  | Bytes |TempSpc| Cost (%CPU)| Time     |                                                
------------------------------------------------------------------------------------                                                
|   0 | SELECT STATEMENT    |      | 68395 |  2738K|       |  1042   (1)| 00:00:13 |                                                
|*  1 |  VIEW               |      | 68395 |  2738K|       |  1042   (1)| 00:00:13 |                                                
|   2 |   WINDOW SORT       |      | 68395 |  2938K|  3784K|  1042   (1)| 00:00:13 |                                                
|   3 |    TABLE ACCESS FULL| T1   | 68395 |  2938K|       |   274   (1)| 00:00:04 |                                                
------------------------------------------------------------------------------------                                                
                                                                                                                                    
Predicate Information (identified by operation id):                                                                                 
---------------------------------------------------                                                                                 
                                                                                                                                    
   1 - filter("OBJECT_NAME"='DUAL')                                                                                                 
                                                                                                                                    
Note                                                                                                                                
-----                                                                                                                               
   - dynamic sampling used for this statement (level=11)                                                                            


Statistics
----------------------------------------------------------                                                                          
          0  recursive calls                                                                                                        
          0  db block gets                                                                                                          
        982  consistent gets                                                                                                        
          0  physical reads                                                                                                         
          0  redo size                                                                                                              
        753  bytes sent via SQL*Net to client                                                                                       
        500  bytes received via SQL*Net from client                                                                                 
          2  SQL*Net roundtrips to/from client                                                                                      
          1  sorts (memory)                                                                                                         
          0  sorts (disk)                                                                                                           
          2  rows processed                                                                                                         

SQL> 
SQL> set autotrace off
SQL> 
SQL> connect bing/##########
Connected.
SQL> 
SQL> set echo on linesize 132
SQL> 
SQL> alter session set optimizer_dynamic_sampling=11;

Session altered.

SQL> 
SQL> EXPLAIN PLAN FOR
  2  SELECT object_type
  3  	  , object_name
  4  	  , object_rank
  5  FROM (
  6  	       SELECT object_type
  7  		    , object_name
  8  		    , ROW_NUMBER() OVER (PARTITION BY object_type ORDER BY created) object_rank
  9  	       FROM   t1
 10  )
 11  WHERE object_name = 'DUAL';

Explained.

SQL> SELECT * FROM TABLE(dbms_xplan.display);

PLAN_TABLE_OUTPUT                                                                                                                   
------------------------------------------------------------------------------------------------------------------------------------
Plan hash value: 2273146475                                                                                                         
                                                                                                                                    
------------------------------------------------------------------------------------                                                
| Id  | Operation           | Name | Rows  | Bytes |TempSpc| Cost (%CPU)| Time     |                                                
------------------------------------------------------------------------------------                                                
|   0 | SELECT STATEMENT    |      | 68395 |  2738K|       |  1042   (1)| 00:00:13 |                                                
|*  1 |  VIEW               |      | 68395 |  2738K|       |  1042   (1)| 00:00:13 |                                                
|   2 |   WINDOW SORT       |      | 68395 |  2938K|  3784K|  1042   (1)| 00:00:13 |                                                
|   3 |    TABLE ACCESS FULL| T1   | 68395 |  2938K|       |   274   (1)| 00:00:04 |                                                
------------------------------------------------------------------------------------                                                
                                                                                                                                    

PLAN_TABLE_OUTPUT                                                                                                                   
------------------------------------------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):                                                                                 
---------------------------------------------------                                                                                 
                                                                                                                                    
   1 - filter("OBJECT_NAME"='DUAL')                                                                                                 
                                                                                                                                    
Note                                                                                                                                
-----                                                                                                                               
   - dynamic sampling used for this statement (level=11)                                                                            

19 rows selected.

SQL> 
SQL> set autotrace on
SQL> 
SQL> select object_type, object_name
  2  from
  3  (select object_type, object_name from t1)
  4  where object_name = 'DUAL';

OBJECT_TYPE         OBJECT_NAME                                                                                                     
------------------- ------------------------------                                                                                  
TABLE               DUAL                                                                                                            
SYNONYM             DUAL                                                                                                            


Execution Plan
----------------------------------------------------------                                                                          
Plan hash value: 3617692013                                                                                                         
                                                                                                                                    
--------------------------------------------------------------------------                                                          
| Id  | Operation         | Name | Rows  | Bytes | Cost (%CPU)| Time     |                                                          
--------------------------------------------------------------------------                                                          
|   0 | SELECT STATEMENT  |      |     2 |    72 |   274   (1)| 00:00:04 |                                                          
|*  1 |  TABLE ACCESS FULL| T1   |     2 |    72 |   274   (1)| 00:00:04 |                                                          
--------------------------------------------------------------------------                                                          
                                                                                                                                    
Predicate Information (identified by operation id):                                                                                 
---------------------------------------------------                                                                                 
                                                                                                                                    
   1 - filter("OBJECT_NAME"='DUAL')                                                                                                 
                                                                                                                                    
Note                                                                                                                                
-----                                                                                                                               
   - dynamic sampling used for this statement (level=11)                                                                            


Statistics
----------------------------------------------------------                                                                          
          0  recursive calls                                                                                                        
          0  db block gets                                                                                                          
        983  consistent gets                                                                                                        
          0  physical reads                                                                                                         
          0  redo size                                                                                                              
        674  bytes sent via SQL*Net to client                                                                                       
        499  bytes received via SQL*Net from client                                                                                 
          2  SQL*Net roundtrips to/from client                                                                                      
          0  sorts (memory)                                                                                                         
          0  sorts (disk)                                                                                                           
          2  rows processed                                                                                                         

SQL> 
SQL> select object_type, object_name, object_rank
  2  from
  3  (select object_type, object_name, row_number() over (partition by object_type order by created) object_rank from t1)
  4  where object_name = 'DUAL';

OBJECT_TYPE         OBJECT_NAME                    OBJECT_RANK                                                                      
------------------- ------------------------------ -----------                                                                      
SYNONYM             DUAL                                     1                                                                      
TABLE               DUAL                                    43                                                                      


Execution Plan
----------------------------------------------------------                                                                          
Plan hash value: 2273146475                                                                                                         
                                                                                                                                    
------------------------------------------------------------------------------------                                                
| Id  | Operation           | Name | Rows  | Bytes |TempSpc| Cost (%CPU)| Time     |                                                
------------------------------------------------------------------------------------                                                
|   0 | SELECT STATEMENT    |      | 68395 |  2738K|       |  1042   (1)| 00:00:13 |                                                
|*  1 |  VIEW               |      | 68395 |  2738K|       |  1042   (1)| 00:00:13 |                                                
|   2 |   WINDOW SORT       |      | 68395 |  2938K|  3784K|  1042   (1)| 00:00:13 |                                                
|   3 |    TABLE ACCESS FULL| T1   | 68395 |  2938K|       |   274   (1)| 00:00:04 |                                                
------------------------------------------------------------------------------------                                                
                                                                                                                                    
Predicate Information (identified by operation id):                                                                                 
---------------------------------------------------                                                                                 
                                                                                                                                    
   1 - filter("OBJECT_NAME"='DUAL')                                                                                                 
                                                                                                                                    
Note                                                                                                                                
-----                                                                                                                               
   - dynamic sampling used for this statement (level=11)                                                                            


Statistics
----------------------------------------------------------                                                                          
          0  recursive calls                                                                                                        
          0  db block gets                                                                                                          
        982  consistent gets                                                                                                        
          0  physical reads                                                                                                         
          0  redo size                                                                                                              
        753  bytes sent via SQL*Net to client                                                                                       
        499  bytes received via SQL*Net from client                                                                                 
          2  SQL*Net roundtrips to/from client                                                                                      
          1  sorts (memory)                                                                                                         
          0  sorts (disk)                                                                                                           
          2  rows processed                                                                                                         

SQL> 
SQL> set autotrace off
SQL> 
SQL> spool off

In both attempts (gathering table stats and gathering schema stats) the dynamic sampling setting was changed from 2 (the default for this database) to 11, and both autotrace and dbms_xplan were used to generate the execution plans. To clear out the environment set by autotrace the user was logged in again before running any SQL statements. No change to the results was noticed (meaning that the generated plan didn’t change using one method over the other) so autotrace had no ‘adverse’ affect on the outcome. Oracle has recently proclaimed this as a bug:


Bug 20591362 - WRONG CARDINALITY WHEN ANALYTICS FUNCTION IS USED

which can be found on MOS. In a nutshell the CBO ignores the selectivity of the WHERE clause when an analytic fuunction is introduced into a query and, as a result, computes the incorrect cardinality.

On a side note if your 11.2.0.x installation does not include Oracle JVM you can run afoul of even stranger numbers:


BANNER                                                                                                                              
--------------------------------------------------------------------------------                                                    
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production                                                        
PL/SQL Release 11.2.0.4.0 - Production                                                                                              
CORE	11.2.0.4.0	Production                                                                                                          
TNS for 64-bit Windows: Version 11.2.0.4.0 - Production                                                                             
NLSRTL Version 11.2.0.4.0 - Production                                                                                              

SQL> 
SQL> create table t1
  2  as
  3  select * From all_objects;

Table created.

SQL> 
SQL> exec dbms_stats.gather_table_stats(user, 'T1')

PL/SQL procedure successfully completed.

SQL> 
SQL> EXPLAIN PLAN FOR
  2  SELECT object_type
  3  	  , object_name
  4  FROM (
  5  	       SELECT object_type
  6  		    , object_name
  7  	       FROM   t1
  8  )
  9  WHERE object_name = 'DUAL';

Explained.

SQL> SELECT * FROM TABLE(dbms_xplan.display);

PLAN_TABLE_OUTPUT                                                                                                                   
------------------------------------------------------------------------------------------------------------------------------------
Plan hash value: 3617692013                                                                                                         
                                                                                                                                    
--------------------------------------------------------------------------                                                          
| Id  | Operation         | Name | Rows  | Bytes | Cost (%CPU)| Time     |                                                          
--------------------------------------------------------------------------                                                          
|   0 | SELECT STATEMENT  |      |     1 |    26 |    56   (0)| 00:00:01 |                                                          
|*  1 |  TABLE ACCESS FULL| T1   |     1 |    26 |    56   (0)| 00:00:01 |                                                          
--------------------------------------------------------------------------                                                          
                                                                                                                                    
Predicate Information (identified by operation id):                                                                                 
---------------------------------------------------                                                                                 

PLAN_TABLE_OUTPUT                                                                                                                   
------------------------------------------------------------------------------------------------------------------------------------
                                                                                                                                    
   1 - filter("OBJECT_NAME"='DUAL')                                                                                                 

13 rows selected.

SQL> 
SQL> set autotrace on
SQL> 
SQL> select object_type, object_name
  2  from
  3  (select object_type, object_name from t1)
  4  where object_name = 'DUAL';

OBJECT_TYPE         OBJECT_NAME                                                                                                     
------------------- ------------------------------                                                                                  
TABLE               DUAL                                                                                                            
SYNONYM             DUAL                                                                                                            


Execution Plan
----------------------------------------------------------                                                                          
Plan hash value: 3617692013                                                                                                         
                                                                                                                                    
--------------------------------------------------------------------------                                                          
| Id  | Operation         | Name | Rows  | Bytes | Cost (%CPU)| Time     |                                                          
--------------------------------------------------------------------------                                                          
|   0 | SELECT STATEMENT  |      |     1 |    26 |    56   (0)| 00:00:01 |                                                          
|*  1 |  TABLE ACCESS FULL| T1   |     1 |    26 |    56   (0)| 00:00:01 |                                                          
--------------------------------------------------------------------------                                                          
                                                                                                                                    
Predicate Information (identified by operation id):                                                                                 
---------------------------------------------------                                                                                 
                                                                                                                                    
   1 - filter("OBJECT_NAME"='DUAL')                                                                                                 


Statistics
----------------------------------------------------------                                                                          
          1  recursive calls                                                                                                        
          0  db block gets                                                                                                          
        191  consistent gets                                                                                                        
          0  physical reads                                                                                                         
          0  redo size                                                                                                              
        674  bytes sent via SQL*Net to client                                                                                       
        500  bytes received via SQL*Net from client                                                                                 
          2  SQL*Net roundtrips to/from client                                                                                      
          0  sorts (memory)                                                                                                         
          0  sorts (disk)                                                                                                           
          2  rows processed                                                                                                         

SQL> 
SQL> select object_type, object_name, object_rank
  2  from
  3  (select object_type, object_name, row_number() over (partition by object_type order by created) object_rank from t1)
  4  where object_name = 'DUAL';

OBJECT_TYPE         OBJECT_NAME                    OBJECT_RANK                                                                      
------------------- ------------------------------ -----------                                                                      
SYNONYM             DUAL                                     1                                                                      
TABLE               DUAL                                    43                                                                      


Execution Plan
----------------------------------------------------------                                                                          
Plan hash value: 2273146475                                                                                                         
                                                                                                                                    
------------------------------------------------------------------------------------                                                
| Id  | Operation           | Name | Rows  | Bytes |TempSpc| Cost (%CPU)| Time     |                                                
------------------------------------------------------------------------------------                                                
|   0 | SELECT STATEMENT    |      | 14281 |   571K|       |   190   (2)| 00:00:03 |                                                
|*  1 |  VIEW               |      | 14281 |   571K|       |   190   (2)| 00:00:03 |                                                
|   2 |   WINDOW SORT       |      | 14281 |   474K|   632K|   190   (2)| 00:00:03 |                                                
|   3 |    TABLE ACCESS FULL| T1   | 14281 |   474K|       |    56   (0)| 00:00:01 |                                                
------------------------------------------------------------------------------------                                                
                                                                                                                                    
Predicate Information (identified by operation id):                                                                                 
---------------------------------------------------                                                                                 
                                                                                                                                    
   1 - filter("OBJECT_NAME"='DUAL')                                                                                                 


Statistics
----------------------------------------------------------                                                                          
          1  recursive calls                                                                                                        
          0  db block gets                                                                                                          
        190  consistent gets                                                                                                        
          0  physical reads                                                                                                         
          0  redo size                                                                                                              
        753  bytes sent via SQL*Net to client                                                                                       
        500  bytes received via SQL*Net from client                                                                                 
          2  SQL*Net roundtrips to/from client                                                                                      
          1  sorts (memory)                                                                                                         
          0  sorts (disk)                                                                                                           
          2  rows processed                                                                                                         

SQL> 
SQL> set autotrace off
SQL> 
SQL> connect bing/##########
Connected.
SQL> 
SQL> set echo on linesize 132
SQL> 
SQL> alter session set optimizer_dynamic_sampling=11;

Session altered.

SQL> 
SQL> EXPLAIN PLAN FOR
  2  SELECT object_type
  3  	  , object_name
  4  	  , object_rank
  5  FROM (
  6  	       SELECT object_type
  7  		    , object_name
  8  		    , ROW_NUMBER() OVER (PARTITION BY object_type ORDER BY created) object_rank
  9  	       FROM   t1
 10  )
 11  WHERE object_name = 'DUAL';

Explained.

SQL> SELECT * FROM TABLE(dbms_xplan.display);

PLAN_TABLE_OUTPUT                                                                                                                   
------------------------------------------------------------------------------------------------------------------------------------
Plan hash value: 2273146475                                                                                                         
                                                                                                                                    
------------------------------------------------------------------------------------                                                
| Id  | Operation           | Name | Rows  | Bytes |TempSpc| Cost (%CPU)| Time     |                                                
------------------------------------------------------------------------------------                                                
|   0 | SELECT STATEMENT    |      | 14281 |   571K|       |   190   (2)| 00:00:03 |                                                
|*  1 |  VIEW               |      | 14281 |   571K|       |   190   (2)| 00:00:03 |                                                
|   2 |   WINDOW SORT       |      | 14281 |   474K|   632K|   190   (2)| 00:00:03 |                                                
|   3 |    TABLE ACCESS FULL| T1   | 14281 |   474K|       |    56   (0)| 00:00:01 |                                                
------------------------------------------------------------------------------------                                                
                                                                                                                                    

PLAN_TABLE_OUTPUT                                                                                                                   
------------------------------------------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):                                                                                 
---------------------------------------------------                                                                                 
                                                                                                                                    
   1 - filter("OBJECT_NAME"='DUAL')                                                                                                 
                                                                                                                                    
Note                                                                                                                                
-----                                                                                                                               
   - dynamic sampling used for this statement (level=11)                                                                            

19 rows selected.

SQL> 
SQL> set autotrace on
SQL> 
SQL> select object_type, object_name
  2  from
  3  (select object_type, object_name from t1)
  4  where object_name = 'DUAL';

OBJECT_TYPE         OBJECT_NAME                                                                                                     
------------------- ------------------------------                                                                                  
TABLE               DUAL                                                                                                            
SYNONYM             DUAL                                                                                                            


Execution Plan
----------------------------------------------------------                                                                          
Plan hash value: 3617692013                                                                                                         
                                                                                                                                    
--------------------------------------------------------------------------                                                          
| Id  | Operation         | Name | Rows  | Bytes | Cost (%CPU)| Time     |                                                          
--------------------------------------------------------------------------                                                          
|   0 | SELECT STATEMENT  |      |     1 |    26 |    56   (0)| 00:00:01 |                                                          
|*  1 |  TABLE ACCESS FULL| T1   |     1 |    26 |    56   (0)| 00:00:01 |                                                          
--------------------------------------------------------------------------                                                          
                                                                                                                                    
Predicate Information (identified by operation id):                                                                                 
---------------------------------------------------                                                                                 
                                                                                                                                    
   1 - filter("OBJECT_NAME"='DUAL')                                                                                                 
                                                                                                                                    
Note                                                                                                                                
-----                                                                                                                               
   - dynamic sampling used for this statement (level=11)                                                                            


Statistics
----------------------------------------------------------                                                                          
         11  recursive calls                                                                                                        
          0  db block gets                                                                                                          
        385  consistent gets                                                                                                        
          0  physical reads                                                                                                         
          0  redo size                                                                                                              
        674  bytes sent via SQL*Net to client                                                                                       
        499  bytes received via SQL*Net from client                                                                                 
          2  SQL*Net roundtrips to/from client                                                                                      
          0  sorts (memory)                                                                                                         
          0  sorts (disk)                                                                                                           
          2  rows processed                                                                                                         

SQL> 
SQL> select object_type, object_name, object_rank
  2  from
  3  (select object_type, object_name, row_number() over (partition by object_type order by created) object_rank from t1)
  4  where object_name = 'DUAL';

OBJECT_TYPE         OBJECT_NAME                    OBJECT_RANK                                                                      
------------------- ------------------------------ -----------                                                                      
SYNONYM             DUAL                                     1                                                                      
TABLE               DUAL                                    43                                                                      


Execution Plan
----------------------------------------------------------                                                                          
Plan hash value: 2273146475                                                                                                         
                                                                                                                                    
------------------------------------------------------------------------------------                                                
| Id  | Operation           | Name | Rows  | Bytes |TempSpc| Cost (%CPU)| Time     |                                                
------------------------------------------------------------------------------------                                                
|   0 | SELECT STATEMENT    |      | 14281 |   571K|       |   190   (2)| 00:00:03 |                                                
|*  1 |  VIEW               |      | 14281 |   571K|       |   190   (2)| 00:00:03 |                                                
|   2 |   WINDOW SORT       |      | 14281 |   474K|   632K|   190   (2)| 00:00:03 |                                                
|   3 |    TABLE ACCESS FULL| T1   | 14281 |   474K|       |    56   (0)| 00:00:01 |                                                
------------------------------------------------------------------------------------                                                
                                                                                                                                    
Predicate Information (identified by operation id):                                                                                 
---------------------------------------------------                                                                                 
                                                                                                                                    
   1 - filter("OBJECT_NAME"='DUAL')                                                                                                 
                                                                                                                                    
Note                                                                                                                                
-----                                                                                                                               
   - dynamic sampling used for this statement (level=11)                                                                            


Statistics
----------------------------------------------------------                                                                          
          7  recursive calls                                                                                                        
          0  db block gets                                                                                                          
        196  consistent gets                                                                                                        
          0  physical reads                                                                                                         
          0  redo size                                                                                                              
        753  bytes sent via SQL*Net to client                                                                                       
        499  bytes received via SQL*Net from client                                                                                 
          2  SQL*Net roundtrips to/from client                                                                                      
          1  sorts (memory)                                                                                                         
          0  sorts (disk)                                                                                                           
          2  rows processed                                                                                                         

SQL> 
SQL> set autotrace off
SQL> 
SQL> exec dbms_stats.gather_schema_stats(user)

PL/SQL procedure successfully completed.

SQL> 
SQL> EXPLAIN PLAN FOR
  2  SELECT object_type
  3  	  , object_name
  4  FROM (
  5  	       SELECT object_type
  6  		    , object_name
  7  	       FROM   t1
  8  )
  9  WHERE object_name = 'DUAL';

Explained.

SQL> SELECT * FROM TABLE(dbms_xplan.display);

PLAN_TABLE_OUTPUT                                                                                                                   
------------------------------------------------------------------------------------------------------------------------------------
Plan hash value: 3617692013                                                                                                         
                                                                                                                                    
--------------------------------------------------------------------------                                                          
| Id  | Operation         | Name | Rows  | Bytes | Cost (%CPU)| Time     |                                                          
--------------------------------------------------------------------------                                                          
|   0 | SELECT STATEMENT  |      |     1 |    26 |    56   (0)| 00:00:01 |                                                          
|*  1 |  TABLE ACCESS FULL| T1   |     1 |    26 |    56   (0)| 00:00:01 |                                                          
--------------------------------------------------------------------------                                                          
                                                                                                                                    
Predicate Information (identified by operation id):                                                                                 
---------------------------------------------------                                                                                 

PLAN_TABLE_OUTPUT                                                                                                                   
------------------------------------------------------------------------------------------------------------------------------------
                                                                                                                                    
   1 - filter("OBJECT_NAME"='DUAL')                                                                                                 
                                                                                                                                    
Note                                                                                                                                
-----                                                                                                                               
   - dynamic sampling used for this statement (level=11)                                                                            

17 rows selected.

SQL> 
SQL> set autotrace on
SQL> 
SQL> select object_type, object_name
  2  from
  3  (select object_type, object_name from t1)
  4  where object_name = 'DUAL';

OBJECT_TYPE         OBJECT_NAME                                                                                                     
------------------- ------------------------------                                                                                  
TABLE               DUAL                                                                                                            
SYNONYM             DUAL                                                                                                            


Execution Plan
----------------------------------------------------------                                                                          
Plan hash value: 3617692013                                                                                                         
                                                                                                                                    
--------------------------------------------------------------------------                                                          
| Id  | Operation         | Name | Rows  | Bytes | Cost (%CPU)| Time     |                                                          
--------------------------------------------------------------------------                                                          
|   0 | SELECT STATEMENT  |      |     1 |    26 |    56   (0)| 00:00:01 |                                                          
|*  1 |  TABLE ACCESS FULL| T1   |     1 |    26 |    56   (0)| 00:00:01 |                                                          
--------------------------------------------------------------------------                                                          
                                                                                                                                    
Predicate Information (identified by operation id):                                                                                 
---------------------------------------------------                                                                                 
                                                                                                                                    
   1 - filter("OBJECT_NAME"='DUAL')                                                                                                 
                                                                                                                                    
Note                                                                                                                                
-----                                                                                                                               
   - dynamic sampling used for this statement (level=11)                                                                            


Statistics
----------------------------------------------------------                                                                          
          0  recursive calls                                                                                                        
          0  db block gets                                                                                                          
        191  consistent gets                                                                                                        
          0  physical reads                                                                                                         
          0  redo size                                                                                                              
        674  bytes sent via SQL*Net to client                                                                                       
        499  bytes received via SQL*Net from client                                                                                 
          2  SQL*Net roundtrips to/from client                                                                                      
          0  sorts (memory)                                                                                                         
          0  sorts (disk)                                                                                                           
          2  rows processed                                                                                                         

SQL> 
SQL> select object_type, object_name, object_rank
  2  from
  3  (select object_type, object_name, row_number() over (partition by object_type order by created) object_rank from t1)
  4  where object_name = 'DUAL';

OBJECT_TYPE         OBJECT_NAME                    OBJECT_RANK                                                                      
------------------- ------------------------------ -----------                                                                      
SYNONYM             DUAL                                     1                                                                      
TABLE               DUAL                                    43                                                                      


Execution Plan
----------------------------------------------------------                                                                          
Plan hash value: 2273146475                                                                                                         
                                                                                                                                    
------------------------------------------------------------------------------------                                                
| Id  | Operation           | Name | Rows  | Bytes |TempSpc| Cost (%CPU)| Time     |                                                
------------------------------------------------------------------------------------                                                
|   0 | SELECT STATEMENT    |      | 14281 |   571K|       |   190   (2)| 00:00:03 |                                                
|*  1 |  VIEW               |      | 14281 |   571K|       |   190   (2)| 00:00:03 |                                                
|   2 |   WINDOW SORT       |      | 14281 |   474K|   632K|   190   (2)| 00:00:03 |                                                
|   3 |    TABLE ACCESS FULL| T1   | 14281 |   474K|       |    56   (0)| 00:00:01 |                                                
------------------------------------------------------------------------------------                                                
                                                                                                                                    
Predicate Information (identified by operation id):                                                                                 
---------------------------------------------------                                                                                 
                                                                                                                                    
   1 - filter("OBJECT_NAME"='DUAL')                                                                                                 
                                                                                                                                    
Note                                                                                                                                
-----                                                                                                                               
   - dynamic sampling used for this statement (level=11)                                                                            


Statistics
----------------------------------------------------------                                                                          
          0  recursive calls                                                                                                        
          0  db block gets                                                                                                          
        190  consistent gets                                                                                                        
          0  physical reads                                                                                                         
          0  redo size                                                                                                              
        753  bytes sent via SQL*Net to client                                                                                       
        499  bytes received via SQL*Net from client                                                                                 
          2  SQL*Net roundtrips to/from client                                                                                      
          1  sorts (memory)                                                                                                         
          0  sorts (disk)                                                                                                           
          2  rows processed                                                                                                         

SQL> 
SQL> set autotrace off
SQL> 
SQL> connect bing/##########
Connected.
SQL> 
SQL> set echo on linesize 132
SQL> 
SQL> alter session set optimizer_dynamic_sampling=11;

Session altered.

SQL> 
SQL> EXPLAIN PLAN FOR
  2  SELECT object_type
  3  	  , object_name
  4  	  , object_rank
  5  FROM (
  6  	       SELECT object_type
  7  		    , object_name
  8  		    , ROW_NUMBER() OVER (PARTITION BY object_type ORDER BY created) object_rank
  9  	       FROM   t1
 10  )
 11  WHERE object_name = 'DUAL';

Explained.

SQL> SELECT * FROM TABLE(dbms_xplan.display);

PLAN_TABLE_OUTPUT                                                                                                                   
------------------------------------------------------------------------------------------------------------------------------------
Plan hash value: 2273146475                                                                                                         
                                                                                                                                    
------------------------------------------------------------------------------------                                                
| Id  | Operation           | Name | Rows  | Bytes |TempSpc| Cost (%CPU)| Time     |                                                
------------------------------------------------------------------------------------                                                
|   0 | SELECT STATEMENT    |      | 14281 |   571K|       |   190   (2)| 00:00:03 |                                                
|*  1 |  VIEW               |      | 14281 |   571K|       |   190   (2)| 00:00:03 |                                                
|   2 |   WINDOW SORT       |      | 14281 |   474K|   632K|   190   (2)| 00:00:03 |                                                
|   3 |    TABLE ACCESS FULL| T1   | 14281 |   474K|       |    56   (0)| 00:00:01 |                                                
------------------------------------------------------------------------------------                                                
                                                                                                                                    

PLAN_TABLE_OUTPUT                                                                                                                   
------------------------------------------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):                                                                                 
---------------------------------------------------                                                                                 
                                                                                                                                    
   1 - filter("OBJECT_NAME"='DUAL')                                                                                                 
                                                                                                                                    
Note                                                                                                                                
-----                                                                                                                               
   - dynamic sampling used for this statement (level=11)                                                                            

19 rows selected.

SQL> 
SQL> set autotrace on
SQL> 
SQL> select object_type, object_name
  2  from
  3  (select object_type, object_name from t1)
  4  where object_name = 'DUAL';

OBJECT_TYPE         OBJECT_NAME                                                                                                     
------------------- ------------------------------                                                                                  
TABLE               DUAL                                                                                                            
SYNONYM             DUAL                                                                                                            


Execution Plan
----------------------------------------------------------                                                                          
Plan hash value: 3617692013                                                                                                         
                                                                                                                                    
--------------------------------------------------------------------------                                                          
| Id  | Operation         | Name | Rows  | Bytes | Cost (%CPU)| Time     |                                                          
--------------------------------------------------------------------------                                                          
|   0 | SELECT STATEMENT  |      |     1 |    26 |    56   (0)| 00:00:01 |                                                          
|*  1 |  TABLE ACCESS FULL| T1   |     1 |    26 |    56   (0)| 00:00:01 |                                                          
--------------------------------------------------------------------------                                                          
                                                                                                                                    
Predicate Information (identified by operation id):                                                                                 
---------------------------------------------------                                                                                 
                                                                                                                                    
   1 - filter("OBJECT_NAME"='DUAL')                                                                                                 
                                                                                                                                    
Note                                                                                                                                
-----                                                                                                                               
   - dynamic sampling used for this statement (level=11)                                                                            


Statistics
----------------------------------------------------------                                                                          
          0  recursive calls                                                                                                        
          0  db block gets                                                                                                          
        191  consistent gets                                                                                                        
          0  physical reads                                                                                                         
          0  redo size                                                                                                              
        674  bytes sent via SQL*Net to client                                                                                       
        500  bytes received via SQL*Net from client                                                                                 
          2  SQL*Net roundtrips to/from client                                                                                      
          0  sorts (memory)                                                                                                         
          0  sorts (disk)                                                                                                           
          2  rows processed                                                                                                         

SQL> 
SQL> select object_type, object_name, object_rank
  2  from
  3  (select object_type, object_name, row_number() over (partition by object_type order by created) object_rank from t1)
  4  where object_name = 'DUAL';

OBJECT_TYPE         OBJECT_NAME                    OBJECT_RANK                                                                      
------------------- ------------------------------ -----------                                                                      
SYNONYM             DUAL                                     1                                                                      
TABLE               DUAL                                    43                                                                      


Execution Plan
----------------------------------------------------------                                                                          
Plan hash value: 2273146475                                                                                                         
                                                                                                                                    
------------------------------------------------------------------------------------                                                
| Id  | Operation           | Name | Rows  | Bytes |TempSpc| Cost (%CPU)| Time     |                                                
------------------------------------------------------------------------------------                                                
|   0 | SELECT STATEMENT    |      | 14281 |   571K|       |   190   (2)| 00:00:03 |                                                
|*  1 |  VIEW               |      | 14281 |   571K|       |   190   (2)| 00:00:03 |                                                
|   2 |   WINDOW SORT       |      | 14281 |   474K|   632K|   190   (2)| 00:00:03 |                                                
|   3 |    TABLE ACCESS FULL| T1   | 14281 |   474K|       |    56   (0)| 00:00:01 |                                                
------------------------------------------------------------------------------------                                                
                                                                                                                                    
Predicate Information (identified by operation id):                                                                                 
---------------------------------------------------                                                                                 
                                                                                                                                    
   1 - filter("OBJECT_NAME"='DUAL')                                                                                                 
                                                                                                                                    
Note                                                                                                                                
-----                                                                                                                               
   - dynamic sampling used for this statement (level=11)                                                                            


Statistics
----------------------------------------------------------                                                                          
          0  recursive calls                                                                                                        
          0  db block gets                                                                                                          
        190  consistent gets                                                                                                        
          0  physical reads                                                                                                         
          0  redo size                                                                                                              
        753  bytes sent via SQL*Net to client                                                                                       
        500  bytes received via SQL*Net from client                                                                                 
          2  SQL*Net roundtrips to/from client                                                                                      
          1  sorts (memory)                                                                                                         
          0  sorts (disk)                                                                                                           
          2  rows processed                                                                                                         

SQL> 
SQL> set autotrace off
SQL> 
SQL> spool off

Notice that the number of rows reported in the execution plans for the non-analytic query are half of what they are supposed to be, and the inflated number of rows is roughly one-fourth of the previously reported value. In the absence of the JVM the data dictionary is ‘short’ around 50,000 rows which can skew the selectivity down to around 1 for the example query. Installing the JVM, using dbca, fixes that issue.

Until this bug is fixed by a one-off patch pay close attention to the row estimates produced for queries using analytic functions; at the least this little ‘inconvenience’ could make troubleshooting performance issues difficult. You may need to eliminate the analytic function to reveal a better estimate of the number of rows returned so you’re not chasing an issue that doesn’t really exist.

Numbers never lie, until they do.

February 23, 2015

Merge, Right?

Filed under: General,Performance — dfitzjarrell @ 15:33

"There are no wrong roads to anywhere."
 - Norton Juster, The Phantom Tollbooth 

Oracle can do some, well, strange things when fixing issues caused by the optimizer. For 10g releases up to 10.2.0.x Oracle chose to silently ignore a message and eliminate an outer join on the MERGE statement under certain conditions. Let’s examine this and see exactly what Oracle implements.

Occasionally in a 10046 trace file generated from 10g releases prior to 10.2.0.x and procesed by the tkprof utility you may see something similar to this:


SELECT 1 FROM DUAL WHERE <some condition here>
 
Error encountered: ORA-00904

where <some condition here> is the ‘ON’ condition from a MERGE statement executed by that session. That seems odd, I know, and Oracle silently ignores that ORA-00904 condition as it is never displayed when executing the MERGE. Why would Oracle do this? Looking at the 10053 trace file generated from the following example:


SQL> merge into t1_merge t1
  2          using (select 1 merged_key, lpad('y', 21, 'y') xtra  from dual) t2
  3          on (t1.merge_key = t2.merged_key)
  4  when matched then update set t1.xtra_stuff = t2.xtra
  5  when not matched then insert (merge_key, more_txt, xtra_stuff) values (t2.merged_key, t2.merged_key, t2.xtra);

1 row merged.

SQL>

we see the following:


PARSE ERROR #5:len=36 dep=1 uid=60 oct=3 lid=60 tim=1266258229533447 err=904
SELECT 1 FROM DUAL WHERE t1.merge_key = t2.merged_key
CLOSE #5:c=0,e=4,dep=1,type=0,tim=1266258229533578
Registered qb: MRG$1 0x84379c (PARSER)
---------------------
QUERY BLOCK SIGNATURE
---------------------
...
====================== END SQL Statement Dump ======================
=====================
PARSING IN CURSOR #1 len=302 dep=0 uid=1158 oct=189 lid=1158 tim=33189168479100 hv=1190985157 ad='1ec71da8'
merge into t1_merge t1
        using (select 1 merged_key, lpad('y', 21, 'y') xtra  from dual) t2
        on (t1.merge_key = t2.merged_key)
when matched then update set t1.xtra_stuff = t2.xtra
when not matched then insert (merge_key, more_txt, xtra_stuff) values (t2.merged_key, t2.merged_key, t2.xtra)
END OF STMT
PARSE #1:c=40000,e=51569,p=4,cr=29,cu=0,mis=1,r=0,dep=0,og=2,tim=33189168479094
BINDS #1:
EXEC #1:c=10000,e=558,p=0,cr=7,cu=3,mis=0,r=1,dep=0,og=2,tim=33189168479778
WAIT #1: nam='SQL*Net message to client' ela= 1 driver id=1650815232 #bytes=1 p3=0 obj#=2135111 tim=33189168479862
WAIT #1: nam='SQL*Net message from client' ela= 419 driver id=1650815232 #bytes=1 p3=0 obj#=2135111 tim=33189168480324
STAT #1 id=1 cnt=2 pid=0 pos=1 obj=0 op='MERGE  T1_MERGE (cr=7 pr=0 pw=0 time=425 us)'
STAT #1 id=2 cnt=1 pid=1 pos=1 obj=0 op='VIEW  (cr=7 pr=0 pw=0 time=178 us)'
STAT #1 id=3 cnt=1 pid=2 pos=1 obj=0 op='NESTED LOOPS OUTER (cr=7 pr=0 pw=0 time=175 us)'
STAT #1 id=4 cnt=1 pid=3 pos=1 obj=0 op='FAST DUAL  (cr=0 pr=0 pw=0 time=1 us)'
STAT #1 id=5 cnt=1 pid=3 pos=2 obj=2135111 op='TABLE ACCESS FULL T1_MERGE (cr=7 pr=0 pw=0 time=170 us)'

Notice there is no outer join in the plan steps. Additionally if we look at the outline data we see:


  /*+
    BEGIN_OUTLINE_DATA
      IGNORE_OPTIM_EMBEDDED_HINTS
      OPTIMIZER_FEATURES_ENABLE('10.2.0.4')
      FIRST_ROWS(10)
      OUTLINE_LEAF(@"SEL$5428C7F1")
      MERGE(@"SEL$2")
      MERGE(@"SEL$3")
      OUTLINE_LEAF(@"SEL$4")
      OUTLINE_LEAF(@"MRG$1")
      OUTLINE(@"SEL$1")
      OUTLINE(@"SEL$2")
      NO_ACCESS(@"MRG$1" "from$_subquery$_008"@"MRG$1")
      NO_ACCESS(@"MRG$1" "T2"@"MRG$1")
      FULL(@"MRG$1" "T1"@"MRG$1")
      LEADING(@"MRG$1" "from$_subquery$_008"@"MRG$1" "T2"@"MRG$1" "T1"@"MRG$1")
      USE_MERGE_CARTESIAN(@"MRG$1" "T2"@"MRG$1")
      USE_MERGE_CARTESIAN(@"MRG$1" "T1"@"MRG$1")
    END_OUTLINE_DATA
  */

The outline data in these cases shows the outer join isn’t used as Oracle re-writes the query into this:


merge into t1
using (select 1 merged_key, lpad('y', 21, 'y') xtra from dual) t2
on (1 = 0)
when matched then update set t1.xtra = t2.xtra
when not matched then insert (merge_key, more_txt, xtra_stuff) values (t2.merged_key, t2.merged_key, t2.xtra);

Inerestingly the reason for this behavior is available in the online documentationL


To insert all of the source rows into the table, you can use a constant filter predicate in the ON clause condition. An example of a 
constant filter predicate is ON (0=1). Oracle Database recognizes such a predicate and makes an unconditional insert of all source 
rows into the table. This approach is different from omitting the merge_update_clause. In that case, the database still must perform 
a join. With constant filter predicate, no join is performed.

This appears to be fixed in 10.2.0.x and later releases as the ORA-00904 error does not occur in either of the trace files. MOS document id 244055.1 lists this modification as an ‘enhancement’ in 10g. The document also lists the bug that requires this work-around is fixed in 10.1.0.1.0. Even though 10g in all its forms is an old release there still may be sites using it.

In 11.2.0.4 this same construct generates no ORA-00904 error, nor does the ‘dummy’ query appear. Notice that the join is restored as evidenced by the execution plan:


----- Current SQL Statement for this session (sql_id=5zv0cdsdp0bf4) -----
merge into t1_merge t1
	using (select 1 merged_key, lpad('y', 21, 'y') xtra  from dual) t2
	on (t1.merge_key = t2.merged_key)
when matched then update set t1.xtra_stuff = t2.xtra
when not matched then insert (merge_key, more_txt, xtra_stuff) values (t2.merged_key, t2.merged_key, t2.xtra)
sql_text_length=289
sql=merge into t1_merge t1
	using (select 1 merged_key, lpad('y', 21, 'y') xtra  from dual) t2
	on (t1.merge_key = t2.merged_key)
when matched then update set t1.xtra_stuff = t2.xtra
when not matched then insert (merge_key, more_txt, xtra_stuff) values (t2.mer
sql=ged_key, t2.merged_key, t2.xtra)
----- Explain Plan Dump -----
----- Plan Table -----
 
============
Plan Table
============
----------------------------------------+-----------------------------------+
| Id  | Operation             | Name    | Rows  | Bytes | Cost  | Time      |
----------------------------------------+-----------------------------------+
| 0   | MERGE STATEMENT       |         |       |       |     6 |           |
| 1   |  MERGE                | T1_MERGE|       |       |       |           |
| 2   |   VIEW                |         |       |       |       |           |
| 3   |    NESTED LOOPS OUTER |         |     1 |    31 |     6 |  00:00:01 |
| 4   |     TABLE ACCESS FULL | DUAL    |     1 |     2 |     2 |  00:00:01 |
| 5   |     TABLE ACCESS FULL | T1_MERGE|     1 |    29 |     4 |  00:00:01 |
----------------------------------------+-----------------------------------+
Predicate Information:
----------------------
5 - filter("T1"."MERGE_KEY"=CASE  WHEN (ROWID IS NOT NULL) THEN 1 ELSE 1 END )

The outline data where this is fixed shows the join is used:


  /*+
    BEGIN_OUTLINE_DATA
      IGNORE_OPTIM_EMBEDDED_HINTS
      OPTIMIZER_FEATURES_ENABLE('10.2.0.4')
      DB_VERSION('11.2.0.4')
      ALL_ROWS
      OUTLINE_LEAF(@"SEL$5428C7F1")
      MERGE(@"SEL$2")
      MERGE(@"SEL$3")
      OUTLINE_LEAF(@"SEL$4")
      OUTLINE_LEAF(@"MRG$1")
      OUTLINE(@"SEL$1")
      OUTLINE(@"SEL$2")
      OUTLINE(@"SEL$3")
      NO_ACCESS(@"MRG$1" "from$_subquery$_008"@"MRG$1")
      NO_ACCESS(@"MRG$1" "T2"@"MRG$1")
      FULL(@"MRG$1" "T1"@"MRG$1")
      LEADING(@"MRG$1" "from$_subquery$_008"@"MRG$1" "T2"@"MRG$1" "T1"@"MRG$1")
      USE_MERGE_CARTESIAN(@"MRG$1" "T2"@"MRG$1")
      USE_MERGE_CARTESIAN(@"MRG$1" "T1"@"MRG$1")
      PQ_DISTRIBUTE(@"MRG$1" "T2"@"MRG$1" NONE NONE)
      PQ_DISTRIBUTE(@"MRG$1" "T1"@"MRG$1" NONE NONE)
      FULL(@"SEL$5428C7F1" "DUAL"@"SEL$3")
      FULL(@"SEL$5428C7F1" "T1"@"SEL$2")
      LEADING(@"SEL$5428C7F1" "DUAL"@"SEL$3" "T1"@"SEL$2")
      USE_NL(@"SEL$5428C7F1" "T1"@"SEL$2")
    END_OUTLINE_DATA
  */

Oracle has, apparently, seen the ‘error of its ways’ with this ‘enhancement’ and re-thought how MERGE should be executed. Of course you may be stuck running an older version of 10g and could be affected by this ‘enhancement’. Even migrating to 10.2.0.4 can eliminate this strange query from your MERGE implementations. Know that this isn’t really a performance problem, but it can be disconcerting to see such essentially useless query text in trace files you may generate. Knowing where it comes from hopefully will make it easier to assess performance issues requiring 10046 and 10053 trace files.

It really helps to pay close attention when merging.

February 11, 2015

What’s The Password?

Filed under: General — dfitzjarrell @ 14:29

A very interesting question was posted in one of the forums I participate in:


is there any way how I can revoke the right from a normal user to change it's own password in the database?

This, obviously, sparked a fairly lively thread (which I will not re-post here) regarding security and the underlying reason such a request was made. The user who posted the original question continued in the thread to explain:


Of course the user should access his own data but I'm trying to prevent that the user can change his own password as he likes. It is part of the security awarness of my company 
and I'm searching for a way to realize that.

This raised more questions and commentary, including the fact that not allowing a user to change his or her own password really implements less security because someone else (the DBA) needs to set it. Any time a password needs to be reset by an administrative user security is compromised, in my opinion. Of course it’s going to be necessary at some point for the DBA to intervene and reset a user’s password — usually when said user has forgotten the password. At those times the DBA will reset the password and force the user to change it to something new at the next login, thus preserving security since no one but the affected user will know his or her password. But, some people are hard to convince.


I think it is more safe, when the "stupid" user can't change his password. He has to raise an incident and the DBA then will change the password

This continued on for a while with various forum members adding comments. So CAN a user be prevented from changing his or her own password and still preserve the ability for the DBA to do so? It appears not, as Pete Finnigan reported back in 2008. He tried using a trigger to generate an error whenever an ALTER USER command was submitted. Why ALTER USER? Let’s find out.

When a user is created in Oracle a curious thing happens; that user automatically, and silently, receives ALTER USER privileges for his or her own account, without an explicit grant. There is nothing to revoke, so that won’t disable the functionality:


SQL> revoke alter user from blorbo;
revoke alter user from blorbo
*
ERROR at line 1:
ORA-01952: system privileges not granted to 'BLORBO'


SQL>

The password function uses ALTER USER behind the scenes to effect the password change. Pete thought that using a trigger would solve the dilemma of a user changing his or her own password, and on one level it does. The trigger is simple, really:


SQL> create or replace trigger alter_trigger
  2  before alter
  3  on database
  4  declare
  5  begin
  6  if (ora_dict_obj_type = 'USER') then
  7  raise_application_error(-20010,'you cannot change your own password');
  8  end if;
  9  end;
 10  /

Trigger created.

SQL>

So far, so good, so let’s test this on a sample user account:


SQL> connect blorbo/#########
Connected.
SQL> password
Changing password for BLORBO
Old password:
New password:
Retype new password:
ERROR:
ORA-00604: error occurred at recursive SQL level 1
ORA-20010: you cannot change your own password
ORA-06512: at line 4


Password unchanged
SQL>

Eureka!!! It works!!! Unfortunately it works TOO well, as now SYS and SYSTEM can’t change a user password:


SQL> show user
USER is "SYS"
SQL> alter user blorbo identified by yoopa;
alter user blorbo identified by yoopa
                                *
ERROR at line 1:
ORA-00604: error occurred at recursive SQL level 1
ORA-20010: you cannot change your own password
ORA-06512: at line 4


SQL>

That’s no good; passwords now cannot be changed and when they expire, well, the user loses access to the database which is not the intended result. Dropping the trigger restores the ability of SYS and SYSTEM to effect password changes but also opens the ‘self-service password door’ we tried to close for “security” reasons.

Another possibility was presented by John Watson in that thread, a basic function to be used as the password verify function. He posted this:


create or replace function no_self_change
(username varchar2,
password varchar2,
old_password varchar2)
RETURN boolean IS
begin
if sys_context('userenv','session_user')=username then
return false;
else
return true;
end if;
end;
/

and it does, indeed, work:


SQL> password
Changing password for BLORBO
Old password:
New password:
Retype new password:
ERROR:
ORA-28003: password verification for the specified password failed


Password unchanged
SQL>
SQL> connect / as sysdba
Connected.
SQL> alter user blorbo identified by yimpski;

User altered.

SQL>

So, we see it is possible to affect the ability for a user to change his or her own database password without affecting the ability of the DBA to do so. But, to be honest, preventing a user from changing their own password is not enforcing security, it’s reducing security by allowing another user (the DBA) to know a user’s password. The DBA should be trustworthy but would YOU want your DBA to know your password? I think not.

Passwords are personal, they should never be shared, and choosing strong passwords is a difficult enough task without adding another person to get the job done. Sometimes what seems like a good security idea at the time turns out to be more of a security risk; not allowing a user to change their own password, in my opinion, falls into that category.

So, there.

February 3, 2015

How Can I Compress Thee

Filed under: Exadata,General,Performance — dfitzjarrell @ 10:26

“You can swim all day in the Sea of Knowledge and not get wet.” 
― Norton Juster, The Phantom Tollbooth 

In previous posts compression options have been discussed, and now it’s time to see how Oracle performs basic compression. It isn’t really compression, it’e de-duplication, but it does result in space savings for data that won’t be modified after it’s ‘compressed’. Let’s look at how Oracle saves space with your data.

Oracle de-duplicates the data by finding common strings, tokenizing them and using the token identifier in the string to reduce the row length. So, what does that mean? Looking at an example might help; a table is built and populated as follows:


--
-- Create and populate the table
--
create table comptst(
	tstcol1	varchar2(4),
	tstcol2 varchar2(6),
	tstcol3	number(8,2),
	tstcol4	varchar2(10));

insert into comptst
values ('ZZXZ', 'bbddff', 199.44, 'PENDING');

insert into comptst
values ('ZZXZ', 'ghijkl', 43.08, 'PENDING');

insert into comptst
values ('ZZXZ', 'bbddff', 881.02, 'PENDING');

insert into comptst
values ('ZZXZ', 'bbddff', 54.97, 'PENDING');

commit;

insert into comptst select * From comptst;
insert into comptst select * From comptst;
insert into comptst select * From comptst;
insert into comptst select * From comptst;
insert into comptst select * From comptst;
insert into comptst select * From comptst;
insert into comptst select * From comptst;
insert into comptst select * From comptst;
insert into comptst select * From comptst;
insert into comptst select * From comptst;
insert into comptst select * From comptst;
insert into comptst select * From comptst;
insert into comptst select * From comptst;
insert into comptst select * From comptst;
insert into comptst select * From comptst;
insert into comptst select * From comptst;
insert into comptst select * From comptst;
insert into comptst select * From comptst;
insert into comptst select * From comptst;
insert into comptst select * From comptst;
insert into comptst select * From comptst;
insert into comptst select * From comptst;
insert into comptst select * From comptst;
insert into comptst select * From comptst;
insert into comptst select * From comptst;

commit;
				--
-- Compress the table with BASIC compression
--
alter table comptst compress basic;
alter table comptst move;

[The table was compressed after the data was inserted which required two steps, the first to set the compression level and the second, a table move in place, to actually compress the data. Had the table been built as compressed and direct path inserts used the data would have been compressed without further action.] Since the initial 4-row insert was re-inserted multiple times there is a lot of duplication in the data, and since Oracle de-duplicates rows to produce the effect of compression there should be a lot of data in a block dump indicating this. There is, and the first piece of that data is the following line:


  perm_9ir2[4]={ 0 2 3 1 }

Oracle builds a token table for each data block; this provides a reference for each data string that occurs more than once in the block. Additionally Oacle can re-arrange the column values in that token table so that multiple column values can be turned into a single token and, thus, a single reference. The line shown above indicates what column values map to the table positions in the token table for this block; in this case column 0 maps to the data in table column 0, column 1 maps to the data in table column 2, column 2 maps to data column 3 and column 3 maps to data column 1. Let’s look at the unique data that was inserted:


('ZZXZ', 'bbddff', 199.44, 'PENDING');
('ZZXZ', 'ghijkl', 43.08, 'PENDING');
('ZZXZ', 'bbddff', 881.02, 'PENDING');
('ZZXZ', 'bbddff', 54.97, 'PENDING');

Since these data rows are duplicated in each block every column is a potential compression token. Two values occur in every row, ‘ZZXZ’ and ‘PENDING’, so it should be expected that tokens for those values will be found in each of the compressed data rows. As mentioned previously Oracle builds a token table in each block so there are two tables in this block, the first, starting at offset 0, is the token table that has 7 rows and the second, starting at offset 7, is the actual table data and there are 721 rows:


0x24:pti[0]	nrow=7		offs=0
0x28:pti[1]	nrow=721	offs=7

Oracle has a clue with this implementation of compression and can create a token that includes a data value and a token, from the same token table, to reduce that row length even further. The examples provided here won’t be demonstrating that but know that it is possible. Now let’s look at the first row in this block for the data table:


tab 1, row 0, @0x1f31
tl: 5 fb: --H-FL-- lb: 0x0  cc: 4
col  0: [ 4]  5a 5a 58 5a
col  1: [ 7]  50 45 4e 44 49 4e 47
col  2: [ 6]  62 62 64 64 66 66
col  3: [ 3]  c1 37 62
bindmp: 2c 00 01 04 02

The actual column lengths are supplied between the square brackets for each column; the total length should be the sum of those values plus 7 bytes, 4 of those for the column lengths, one for the lock byte, one for the flag byte and one for the column count. Using that information the total length should be 24 bytes; the block dump provides a different total length of 5, as reported by the tl entry. There is a line at the end of the row dump labeled bindmp (a binary dump of the row contents) revealing the actual contents of those 5 bytes. As expected there is the lock byte (0x2c), the number of columns at this location (0x01) and two bytes representing the token, reporting that 4 columns are in this token and that the reference row in the token table is row 2. So, let’s look at table 0, row 2:


tab 0, row 2, @0x1f5c
tl: 10 fb: --H-FL-- lb: 0x0  cc: 4
col  0: [ 4]  5a 5a 58 5a
col  1: [ 7]  50 45 4e 44 49 4e 47
col  2: [ 6]  62 62 64 64 66 66
col  3: [ 3]  c1 37 62
bindmp: 00 b3 04 04 05 06 cb c1 37 62

It looks almost like the data row, but the total token length is 10 bytes. Looking at the bindmp the first two bytes indicate this token is used 179 times in this block, the third byte indicates that 4 columns are in this token, the two bytes after that report that the first two columns are also tokens, 0x04 and 0x05. Going back to the token table we see that those tokens are:


tab 0, row 4, @0x1f66
tl: 7 fb: --H-FL-- lb: 0x0  cc: 1
col  0: [ 4]  5a 5a 58 5a
bindmp: 00 04 cc 5a 5a 58 5a
tab 0, row 5, @0x1f76
tl: 10 fb: --H-FL-- lb: 0x0  cc: 1
col  0: [ 7]  50 45 4e 44 49 4e 47
bindmp: 00 04 cf 50 45 4e 44 49 4e 47

These are single-column tokens, and each is used 4 times in this block. This is how Oracle reduced the row length from 24 bytes to 5 to save block space. Working through the block dump it’s now possible to re-construct the 24 bytes of data the row originally contained even though it now is only 5 bytes in length.


"The way you see things depends a great deal on where you look at them from."
 - Norton Juster, The Phantom Tollbooth 

We see that Oracle doesn’t actually compress data, it replaces duplicate values with tokens and, through those tokens, reconstructs the data at query time by using the row directory and the actual row pieces in each block. Depending on the select list some tokens won’t be accessed if that data isn’t required. Of course all of this re-constructing can be expensive at the CPU level, and for full table scans of large tables performance can be an issue, especially with the “cache buffers chains” latch because Oracle is performing fewer “consistent gets – examination”. This is because Oracle has to pin blocks for a longer period due to the reconstruction. On the plus side the number of physical reads can decrease since the data is in a smaller number of blocks and can stay in the cache longer. Using basic compression is a trade-off between size and performance, and for extremely large tables or in cases where the compression savings are quite large (meaning the data is compressed more) queries may become CPU-intensive rather than I/O intensive. The good and the bad need to be weighed carefully when making the decision to use compression; choose wisely. Space is relatively inexpensive when compared to end-user satisfaction. The DBA’s idea of performance and the end-users ideas of performance use different criteria, and it’s really the end-users idea that should take precendence.

Anyone up for a swim?

January 13, 2015

Time Is On My Side, Maybe

Filed under: General — dfitzjarrell @ 08:23

A recent MOS document discusses, albeit briefly, an issue with AWR reports that’s been going on since 10.2.0, namely that the elapsed time numbers for queries executed in parallel are considerably greater than the actual elapsed clock time. Let’s look at why that is and what can be done about it.

AWR reports were a tremendous improvement over Statspack reports, primarily due to the depth and breadth of the sample data collected. A more detailed report, with extended metrics, provides a better look at what the database is doing during a given interval. Unfortunately this detail can create unusual information, especially in the SQL Elapsed Time section. For ‘normal’ queries (read that as ‘non-parallel’) the elapsed time is usually pretty accurate. Add in parallel query slaves and the overal elapsed time can grow to several times the actual clock time the query or statement consumed. Let’s set up an example, running in parallel, and see what the AWR report provides.

Let’s create a copy of EMP that has a non-0 degree of parallelism:

SQL> CREATE TABLE EMP
  2         (EMPNO NUMBER(4) NOT NULL,
  3          ENAME VARCHAR2(10),
  4          JOB VARCHAR2(9),
  5          MGR NUMBER(4),
  6          HIREDATE DATE,
  7          SAL NUMBER(7, 2),
  8          COMM NUMBER(7, 2),
  9          DEPTNO NUMBER(2))
 10  parallel 4;

Table created.

SQL>

Now let’s make it a fairly large table:

SQL> begin
  2  for i in 1..20 loop
  3     insert into emp
  4     select * from emp;
  5  end loop;
  6
  7  commit;
  8
  9  end;
 10  /

PL/SQL procedure successfully completed.

SQL> select count(*)
  2  from emp;

  COUNT(*)
----------
  14680064

SQL>

Verifying that parallel is being used to process queries:


SQL>> select *
  2  from emp
  3  where empno < 7400;

     EMPNO ENAME      JOB              MGR HIREDATE         SAL       COMM     DEPTNO
---------- ---------- --------- ---------- --------- ---------- ---------- ----------
      7369 SMITH      CLERK           7902 17-DEC-80        800                    20
      7369 SMITH      CLERK           7902 17-DEC-80        800                    20
      7369 SMITH      CLERK           7902 17-DEC-80        800                    20
      7369 SMITH      CLERK           7902 17-DEC-80        800                    20
      7369 SMITH      CLERK           7902 17-DEC-80        800                    20
      7369 SMITH      CLERK           7902 17-DEC-80        800                    20
...
      7369 SMITH      CLERK           7902 17-DEC-80        800                    20

1048576 rows selected.


Execution Plan
----------------------------------------------------------
Plan hash value: 2873591275

--------------------------------------------------------------------------------------------------------------
| Id  | Operation            | Name     | Rows  | Bytes | Cost (%CPU)| Time     |    TQ  |IN-OUT| PQ Distrib |
--------------------------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT     |          |  1073K|    89M|  6543   (1)| 00:01:19 |        |      |            |
|   1 |  PX COORDINATOR      |          |       |       |            |          |        |      |            |
|   2 |   PX SEND QC (RANDOM)| :TQ10000 |  1073K|    89M|  6543   (1)| 00:01:19 |  Q1,00 | P->S | QC (RAND)  |
|   3 |    PX BLOCK ITERATOR |          |  1073K|    89M|  6543   (1)| 00:01:19 |  Q1,00 | PCWC |            |
|*  4 |     TABLE ACCESS FULL| EMP      |  1073K|    89M|  6543   (1)| 00:01:19 |  Q1,00 | PCWP |            |
--------------------------------------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   4 - filter("EMPNO"

Now generate an AWR report for the interval in question:

SQL ordered by Elapsed Time

  • Resources reported for PL/SQL code includes the resources used by all SQL statements called by the code.
  • % Total DB Time is the Elapsed Time of the SQL statement divided into the Total Database Time multiplied by 100
  • %Total – Elapsed Time as a percentage of Total DB time
  • %CPU – CPU Time as a percentage of Elapsed Time
  • %IO – User I/O Time as a percentage of Elapsed Time
  • Captured SQL account for 98.0% of Total DB Time (s): 18
  • Captured PL/SQL account for 0.6% of Total DB Time (s): 18
Elapsed Time (s) Executions Elapsed Time per Exec (s) %Total %CPU %IO SQL Id SQL Module SQL Text
10.36 1 10.36 58.70 19.57 73.06 avrt3mgztynzh SQL*Plus select /*+ parallel(12) */ * f…
4.96 1 4.96 28.12 96.15 1.89 572fbaj0fdw2b sqlplus.exe select output from table(dbms_…
0.20 1 0.20 1.11 103.08 0.00 bunssq950snhf   insert into wrh$_sga_target_ad…
0.15 16 0.01 0.87 111.50 0.08 3s1hh8cvfan6w SQL*Plus SELECT PLAN_TABLE_OUTPUT FROM …
0.15 1 0.15 0.87 91.92 0.00 dayq182sk41ks   insert into wrh$_memory_target…
0.15 1 0.15 0.86 92.16 0.00 bm2pwrpcr8ru6   select sga_size s, sga_size_fa…
0.14 16 0.01 0.82 108.24 0.08 g3f3cw3zy5aat SQL*Plus SELECT PLAN_TABLE_OUTPUT FROM …
0.13 16 0.01 0.72 122.57 0.00 f0a1c90fu7jtd SQL*Plus SELECT /*+ opt_param(‘parallel…
0.12 121 0.00 0.69 12.75 0.00 772s25v1y0x8k   select shared_pool_size_for_es…
0.09 1 0.09 0.52 101.63 0.00 1bx8mgs8by25x sqlplus.exe select coalesce( p1s.parameter…


Notice that in this AWR report the elapsed time for our query is 10.36 seconds. The actual elapsed time, reported when timing is on, and the execution plan is:


Elapsed: 00:04:29.31

Execution Plan
----------------------------------------------------------
Plan hash value: 2873591275

--------------------------------------------------------------------------------------------------------------
| Id  | Operation            | Name     | Rows  | Bytes | Cost (%CPU)| Time     |    TQ  |IN-OUT| PQ Distrib |
--------------------------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT     |          |  1073K|    89M|  2181   (1)| 00:00:05 |        |      |         |
|   1 |  PX COORDINATOR      |          |       |       |            |          |        |      |         |
|   2 |   PX SEND QC (RANDOM)| :TQ10000 |  1073K|    89M|  2181   (1)| 00:00:05 |  Q1,00 | P->S | QC (RAND)  |
|   3 |    PX BLOCK ITERATOR |          |  1073K|    89M|  2181   (1)| 00:00:05 |  Q1,00 | PCWC |         |
|*  4 |     TABLE ACCESS FULL| EMP      |  1073K|    89M|  2181   (1)| 00:00:05 |  Q1,00 | PCWP |         |
--------------------------------------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   4 - filter("EMPNO"<=7400)

So why the discrepancy? Why does AWR report a longer elapsed time than actually occurred? Oracle Support addresses this in MOS note 1956033.1 and reports that this behavior is not a bug. According to Oracle Support the elapsed time (taken from the V$SQL view) includes time for the Query Coordinator and the parallel query slaves operating on that query. In a non-parallel execution such aggregation produces results very close, if not exactly matching, the wall clock time. In this case the elapsed time computed for the AWR report is over twice as long as the actual execution time. This doesn’t mean that there were two parallel query slaves in use; the query was hinted to provide a parallelism of 12. Thus, the sum of the elapsed time for the Query Coordinator and the elapsed time for each of the 12 parallel query slaves adds up to 10.36 seconds. If we set the degree for the EMP table to 1 and run a non-hinted query to get a serial execution time the result is:


Elapsed: 00:02:04.08

Execution Plan
----------------------------------------------------------
Plan hash value: 3956160932

--------------------------------------------------------------------------
| Id  | Operation         | Name | Rows  | Bytes | Cost (%CPU)| Time     |
--------------------------------------------------------------------------
|   0 | SELECT STATEMENT  |      |  1073K|    89M| 23573   (1)| 00:04:43 |
|*  1 |  TABLE ACCESS FULL| EMP  |  1073K|    89M| 23573   (1)| 00:04:43 |
--------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   1 - filter("EMPNO"<=7400)

And another AWR report shows:

SQL ordered by Elapsed Time

  • Resources reported for PL/SQL code includes the resources used by all SQL statements called by the code.
  • % Total DB Time is the Elapsed Time of the SQL statement divided into the Total Database Time multiplied by 100
  • %Total – Elapsed Time as a percentage of Total DB time
  • %CPU – CPU Time as a percentage of Elapsed Time
  • %IO – User I/O Time as a percentage of Elapsed Time
  • Captured SQL account for 97.1% of Total DB Time (s): 3
  • Captured PL/SQL account for 37.2% of Total DB Time (s): 3
Elapsed Time (s) Executions Elapsed Time per Exec (s) %Total %CPU %IO SQL Id SQL Module SQL Text
1.62 1 1.62 48.69 93.32 2.70 19duabgggdwaj SQL*Plus select * from emp where empno …
1.07 0   32.07 65.73 18.91 1uk5m5qbzj1vt SQL*Plus BEGIN dbms_workload_repository…
0.22 1 0.22 6.75 104.07 0.00 bunssq950snhf   insert into wrh$_sga_target_ad…
0.18 1 0.18 5.55 75.93 0.14 32b51bs6x2ux7 SQL*Plus EXPLAIN PLAN SET STATEMENT_ID=…
0.17 2 0.08 4.98 47.02 14.20 gpf6csqpdaymp SQL*Plus select * From dba_scheduler_jo…
0.16 1 0.16 4.81 97.32 0.00 dayq182sk41ks   insert into wrh$_memory_target…
0.16 1 0.16 4.77 98.25 0.00 bm2pwrpcr8ru6   select sga_size s, sga_size_fa…
0.10 1 0.10 2.88 0.00 74.78 6ajkhukk78nsr   begin prvt_hdm.auto_execute( :…
0.07 66 0.00 2.23 0.00 96.67 5h7w8ykwtb2xt   INSERT INTO SYS.WRI$_ADV_PARAM…
0.06 1 0.06 1.94 0.00 93.15 f0dtmk7ap93vx DBMS_SCHEDULER DECLARE object_type varchar2(4…
0.06 1 0.06 1.72 27.31 87.39 350myuyx0t1d6   insert into wrh$_tablespace_st…
0.04 2 0.02 1.13 83.07 1.25 3s1hh8cvfan6w SQL*Plus SELECT PLAN_TABLE_OUTPUT FROM …
0.04 2 0.02 1.08 87.09 1.31 g3f3cw3zy5aat SQL*Plus SELECT PLAN_TABLE_OUTPUT FROM …
0.04 34 0.00 1.06 0.00 0.00 772s25v1y0x8k   select shared_pool_size_for_es…

In this report the elapsed time for the serial query (1.62 seconds) closely aligns with the reported elapsed time from SQL*Plus (2.04 seconds).

The question remains: how to get a fairly accurate elapsed time for queries executing in parallel? As demonstrated here using:


set timing on

is one way to get the elapsed time. Another, more accurate, way is to generate a 10046 trace for the query in question. The trace file, and the formatted output from tkprof, will provide the correct elapsed time for the query:


select /*+ parallel(12) */ *
from emp
where empno <= 7400

call     count       cpu    elapsed       disk      query    current        rows
------- ------  -------- ---------- ---------- ---------- ----------  ----------
Parse        1      0.00       0.03          0          1          0           0
Execute      1      0.00       0.18          0         15          0           0
Fetch    69907      2.23       2.36          0          0          0     1048576
------- ------  -------- ---------- ---------- ---------- ----------  ----------
total    69909      2.23       2.58          0         16          0     1048576

Misses in library cache during parse: 1
Optimizer mode: ALL_ROWS
Parsing user id: 184
Number of plan statistics captured: 1

Rows (1st) Rows (avg) Rows (max)  Row Source Operation
---------- ---------- ----------  ---------------------------------------------------
   1048576    1048576    1048576  PX COORDINATOR  (cr=15 pr=0 pw=0 time=1086677 us)
         0          0          0   PX SEND QC (RANDOM) :TQ10000 (cr=0 pr=0 pw=0 time=0 us cost=2181 size=93396936 card=1073528)
         0          0          0    PX BLOCK ITERATOR (cr=0 pr=0 pw=0 time=0 us cost=2181 size=93396936 card=1073528)
         0          0          0     TABLE ACCESS FULL EMP (cr=0 pr=0 pw=0 time=0 us cost=2181 size=93396936 card=1073528)

The elapsed time reported by the 10046 trace is less than that reported by ‘set timing on’ and that’s probably due to the time spent sending output to the display. The actual database elapsed time was under 3 seconds, with the elapsed time reported by SQL*Plue as 4.29 seconds. This is an acceptable discrepancy given that displaying results to a terminal screen takes time. I wouldn’t expect the times to match as two different mechanisms are in use to generate elapsed time between the two tools. They are close enough to know that the AWR report, aggregating time from V$SQL, is reporting inflated figures because of the parallel execution.

Parallel query execution can save time in some cases by using a ‘divide an conquer’ approach to result set processing. It is unfortunate tha the V$SQL performance view contains inflated elapsed times for queries run in parallel. Since Oracle Support does not consider this a bug it’s not going to be changed any time soon. Knowing this makes it easier to understand elapsed times in AWR reports when parallelism is used, and now you have at least two ways to generate fairly accurate elapsed execution times for such queries. It makes it a bit more involved to tune queries using parallism but the effort is worth the time and trouble.

If only my work day could log time the same way.

December 18, 2014

The Best Laid Plans

Filed under: General — dfitzjarrell @ 10:54

In a forum I contribute to the following question was asked:


Can adaptive cursor sharing (ACS) depend on execution order ?

The issue described a relatively basic query that changed execution plans apparently due to the order the query statements were run based on bind variable values. It’s an interesting issue that testing has verified. The tests are reproduced below, in abbreviated form, so let’s look at what was executed and what execution plans were used.

Adaptive Cursor Sharing, for those who aren’t familiar with the term, allows Oracle to bind peek every time a query is run to determine the actual value in the bind variable. Knowing this helps Oracle provide a reasonably performant execution plan and can generate several different plans for the same query text. Oracle then selects from the possible cursors and the associated plans to execute the bind variable query in, hopefully, the most efficient manner. A key to this is current statistics, including histograms, so the optimizer can ‘know’ the data distribution and can better assess an efficient access path. With 12.1.0.2 a bug exists, involving the use of histograms, that affects execution plans for bind variable queries. Examples are provided illustrating the problem relative to non-bind variable queries.

First let’s consider not using bind variables and see what plans are generated. Tables were created and loaded in a way to allow an index scan for one salary value and a full table scan for another salary value. The non-bind-variable results follow:


SQL> 
SQL> --
SQL> -- Create sample tables
SQL> --
SQL> create table emp(
  2  	     empid   number,
  3  	     empnm   varchar2(40),
  4  	     empsal  number,
  5  	     empssn  varchar2(12),
  6  	     constraint emp_pk primary key (empid)
  7  ) ;

Table created.

SQL> 
SQL> create index empsal_idx on emp(empsal) ;

Index created.

SQL> 
SQL> create table emp_dept(
  2  	     empid   number,
  3  	     empdept number,
  4  	     emploc  varchar2(60),
  5  	     constraint emp_dept_pk primary key(empid)
  6  ) ;

Table created.

SQL> 
SQL> create table dept_info(
  2  	     deptnum number,
  3  	     deptnm  varchar2(25),
  4  	     constraint dept_info_pk primary key(deptnum)
  5  ) ;

Table created.

SQL> 
SQL> --
SQL> -- Load sample tables with data
SQL> --
SQL> begin
  2  	     for i in 1..2000000 loop
  3  		     insert into emp
  4  		     values(i, 'Fnarm'||i, (mod(i, 7)+1)*1000, mod(i,10)||mod(i,10)||mod(i,10)||'-'||mod(i,10)||mod(i,10)||'-'||mod(i,10)||mod(i,10)||mod(i,10)||mod(i,10));
  5  		     insert into emp_dept
  6  		     values(i, (mod(i,8)+1)*10, 'Zanzwalla'||(mod(i,8)+1)*10);
  7  		     commit;
  8  	     end loop;
  9  	     update emp
 10  	     set empsal=7000 where empsal in (3000,4000,5000);
 11  	     commit;
 12  	     update emp set empsal=800 where empsal=1000 and empssn not like '9%';
 13  	     commit;
 14  	     insert into dept_info
 15  	     select distinct empdept, case when empdept = 10 then 'SALES'
 16  					   when empdept = 20 then 'PROCUREMENT'
 17  					   when empdept = 30 then 'HR'
 18  					   when empdept = 40 then 'RESEARCH'
 19  					   when empdept = 50 then 'DEVELOPMENT'
 20  					   when empdept = 60 then 'EMPLOYEE RELATIONS'
 21  					   when empdept = 70 then 'FACILITIES'
 22  					   when empdept = 80 then 'FINANCE' end
 23  	     from emp_dept;
 24  
 25  end;
 26  /

PL/SQL procedure successfully completed.

SQL> 
SQL> exec dbms_stats.gather_schema_stats('BING')

PL/SQL procedure successfully completed.

SQL> 
SQL> --
SQL> -- Run query on indexed column
SQL> -- and use index
SQL> --
SQL> -- Generate execution plan
SQL> --
SQL> set autotrace on
SQL> 
SQL> select *
  2  from emp
  3  where empsal = 1000;

     EMPID EMPNM                                        EMPSAL EMPSSN                                                                                 
---------- ---------------------------------------- ---------- ------------                                                                           
       679 Fnarm679                                       1000 999-99-9999                                                                            
       749 Fnarm749                                       1000 999-99-9999                                                                            
       819 Fnarm819                                       1000 999-99-9999                                                                            
       889 Fnarm889                                       1000 999-99-9999                                                                            
       959 Fnarm959                                       1000 999-99-9999                                                                            
...
   1958719 Fnarm1958719                                   1000 999-99-9999                                                                            
   1210839 Fnarm1210839                                   1000 999-99-9999                                                                            
   1210699 Fnarm1210699                                   1000 999-99-9999                                                                            
   1210769 Fnarm1210769                                   1000 999-99-9999                                                                            

28571 rows selected.


Execution Plan
----------------------------------------------------------                                                                                            
Plan hash value: 3668658578                                                                                                                           
                                                                                                                                                      
--------------------------------------------------------------------------------------------------                                                    
| Id  | Operation                           | Name       | Rows  | Bytes | Cost (%CPU)| Time     |                                                    
--------------------------------------------------------------------------------------------------                                                    
|   0 | SELECT STATEMENT                    |            | 28571 |   892K|   911   (1)| 00:00:01 |                                                    
|   1 |  TABLE ACCESS BY INDEX ROWID BATCHED| EMP        | 28571 |   892K|   911   (1)| 00:00:01 |                                                    
|*  2 |   INDEX RANGE SCAN                  | EMPSAL_IDX | 28571 |       |    96   (2)| 00:00:01 |                                                    
--------------------------------------------------------------------------------------------------                                                    
                                                                                                                                                      
Predicate Information (identified by operation id):                                                                                                   
---------------------------------------------------                                                                                                   
                                                                                                                                                      
   2 - access("EMPSAL"=1000)                                                                                                                          


Statistics
----------------------------------------------------------                                                                                            
          1  recursive calls                                                                                                                          
          0  db block gets                                                                                                                            
      17017  consistent gets                                                                                                                          
       6790  physical reads                                                                                                                           
      52960  redo size                                                                                                                                
    1454131  bytes sent via SQL*Net to client                                                                                                         
      21496  bytes received via SQL*Net from client                                                                                                   
       1906  SQL*Net roundtrips to/from client                                                                                                        
          0  sorts (memory)                                                                                                                           
          0  sorts (disk)                                                                                                                             
      28571  rows processed                                                                                                                           

SQL> 
SQL> --
SQL> -- Now use indexed column and NOT use index
SQL> --
SQL> select *
  2  from emp
  3  where emPsal=7000;

     EMPID EMPNM                                        EMPSAL EMPSSN                                                                                 
---------- ---------------------------------------- ---------- ------------                                                                           
       667 Fnarm667                                       7000 777-77-7777                                                                            
       668 Fnarm668                                       7000 888-88-8888                                                                            
       669 Fnarm669                                       7000 999-99-9999                                                                            
       671 Fnarm671                                       7000 111-11-1111                                                                            
       674 Fnarm674                                       7000 444-44-4444                                                                            
...
   1958760 Fnarm1958760                                   7000 000-00-0000                                                                            

1142857 rows selected.


Execution Plan
----------------------------------------------------------                                                                                            
Plan hash value: 3956160932                                                                                                                           
                                                                                                                                                      
--------------------------------------------------------------------------                                                                            
| Id  | Operation         | Name | Rows  | Bytes | Cost (%CPU)| Time     |                                                                            
--------------------------------------------------------------------------                                                                            
|   0 | SELECT STATEMENT  |      |  1142K|    34M|  1507   (8)| 00:00:01 |                                                                            
|*  1 |  TABLE ACCESS FULL| EMP  |  1142K|    34M|  1507   (8)| 00:00:01 |                                                                            
--------------------------------------------------------------------------                                                                            
                                                                                                                                                      
Predicate Information (identified by operation id):                                                                                                   
---------------------------------------------------                                                                                                   
                                                                                                                                                      
   1 - filter("EMPSAL"=7000)                                                                                                                          


Statistics
----------------------------------------------------------                                                                                            
          1  recursive calls                                                                                                                          
          0  db block gets                                                                                                                            
      86286  consistent gets                                                                                                                          
       6384  physical reads                                                                                                                           
          0  redo size                                                                                                                                
   54471009  bytes sent via SQL*Net to client                                                                                                         
     838642  bytes received via SQL*Net from client                                                                                                   
      76192  SQL*Net roundtrips to/from client                                                                                                        
          0  sorts (memory)                                                                                                                           
          0  sorts (disk)                                                                                                                             
    1142857  rows processed                                                                                                                           

SQL> 

The results are as expected, with the query looking for salary value 1000 uses the index (the data volume is small enough to make this the most efficient path) and the query looking for the salary value 7000 uses a full table scan. Things get interesting when the literals are replaced by a bind variable:


SQL> --
SQL> -- Create sample tables
SQL> --
SQL> create table emp(
  2  	     empid   number,
  3  	     empnm   varchar2(40),
  4  	     empsal  number,
  5  	     empssn  varchar2(12),
  6  	     constraint emp_pk primary key (empid)
  7  ) ;

Table created.

SQL> 
SQL> create index empsal_idx on emp(empsal) ;

Index created.

SQL> 
SQL> create table emp_dept(
  2  	     empid   number,
  3  	     empdept number,
  4  	     emploc  varchar2(60),
  5  	     constraint emp_dept_pk primary key(empid)
  6  ) ;

Table created.

SQL> 
SQL> create table dept_info(
  2  	     deptnum number,
  3  	     deptnm  varchar2(25),
  4  	     constraint dept_info_pk primary key(deptnum)
  5  ) ;

Table created.

SQL> 
SQL> --
SQL> -- Load sample tables with data
SQL> --
SQL> begin
  2  	     for i in 1..2000000 loop
  3  		     insert into emp
  4  		     values(i, 'Fnarm'||i, (mod(i, 7)+1)*1000, mod(i,10)||mod(i,10)||mod(i,10)||'-'||mod(i,10)||mod(i,10)||'-'||mod(i,10)||mod(i,10)||mod(i,10)||mod(i,10));
  5  		     insert into emp_dept
  6  		     values(i, (mod(i,8)+1)*10, 'Zanzwalla'||(mod(i,8)+1)*10);
  7  		     commit;
  8  	     end loop;
  9  	     update emp
 10  	     set empsal=7000 where empsal in (3000,4000,5000);
 11  	     commit;
 12  	     update emp set empsal=800 where empsal=1000 and empssn not like '9%';
 13  	     commit;
 14  	     insert into dept_info
 15  	     select distinct empdept, case when empdept = 10 then 'SALES'
 16  					   when empdept = 20 then 'PROCUREMENT'
 17  					   when empdept = 30 then 'HR'
 18  					   when empdept = 40 then 'RESEARCH'
 19  					   when empdept = 50 then 'DEVELOPMENT'
 20  					   when empdept = 60 then 'EMPLOYEE RELATIONS'
 21  					   when empdept = 70 then 'FACILITIES'
 22  					   when empdept = 80 then 'FINANCE' end
 23  	     from emp_dept;
 24  
 25  end;
 26  /

PL/SQL procedure successfully completed.

SQL> 
SQL> exec dbms_stats.gather_schema_stats('BING')

PL/SQL procedure successfully completed.

SQL> 
SQL> --
SQL> -- Run query on indexed column
SQL> -- and use index
SQL> --
SQL> -- Generate execution plan
SQL> --
SQL> variable esal number
SQL> 
SQL> begin
  2  	     :esal:=1000;
  3  end;
  4  /

PL/SQL procedure successfully completed.

SQL> 
SQL> set autotrace on
SQL> 
SQL> select *
  2  from emp
  3  where empsal = :esal;

     EMPID EMPNM                                        EMPSAL EMPSSN                                                                                 
---------- ---------------------------------------- ---------- ------------                                                                           
       889 Fnarm889                                       1000 999-99-9999                                                                            
       959 Fnarm959                                       1000 999-99-9999                                                                            
      1029 Fnarm1029                                      1000 999-99-9999                                                                            
      1099 Fnarm1099                                      1000 999-99-9999                                                                            
        49 Fnarm49                                        1000 999-99-9999                                                                            
       119 Fnarm119                                       1000 999-99-9999                                                                            
...
   1934569 Fnarm1934569                                   1000 999-99-9999                                                                            
   1937439 Fnarm1937439                                   1000 999-99-9999                                                                            
   1937509 Fnarm1937509                                   1000 999-99-9999                                                                            

28571 rows selected.


Execution Plan
----------------------------------------------------------                                                                                            
Plan hash value: 3956160932                                                                                                                           
                                                                                                                                                      
--------------------------------------------------------------------------                                                                            
| Id  | Operation         | Name | Rows  | Bytes | Cost (%CPU)| Time     |                                                                            
--------------------------------------------------------------------------                                                                            
|   0 | SELECT STATEMENT  |      |   400K|    12M|  1544  (10)| 00:00:01 |                                                                            
|*  1 |  TABLE ACCESS FULL| EMP  |   400K|    12M|  1544  (10)| 00:00:01 |                                                                            
--------------------------------------------------------------------------                                                                            
                                                                                                                                                      
Predicate Information (identified by operation id):                                                                                                   
---------------------------------------------------                                                                                                   
                                                                                                                                                      
   1 - filter("EMPSAL"=TO_NUMBER(:ESAL))                                                                                                              


Statistics
----------------------------------------------------------                                                                                            
          1  recursive calls                                                                                                                          
          0  db block gets                                                                                                                            
      16841  consistent gets                                                                                                                          
       6391  physical reads                                                                                                                           
      47780  redo size                                                                                                                                
    1454131  bytes sent via SQL*Net to client                                                                                                         
      21495  bytes received via SQL*Net from client                                                                                                   
       1906  SQL*Net roundtrips to/from client                                                                                                        
          0  sorts (memory)                                                                                                                           
          0  sorts (disk)                                                                                                                             
      28571  rows processed                                                                                                                           

SQL> 
SQL> --
SQL> -- Now use indexed column and NOT use index
SQL> --
SQL> begin
  2  	     :esal:=7000;
  3  end;
  4  /

PL/SQL procedure successfully completed.

SQL> 
SQL> select *
  2  from emp
  3  where empsal=:esal;

     EMPID EMPNM                                        EMPSAL EMPSSN                                                                                 
---------- ---------------------------------------- ---------- ------------                                                                           
       888 Fnarm888                                       7000 888-88-8888                                                                            
       891 Fnarm891                                       7000 111-11-1111                                                                            
       892 Fnarm892                                       7000 222-22-2222                                                                            
       893 Fnarm893                                       7000 333-33-3333                                                                            
...
   1937557 Fnarm1937557                                   7000 777-77-7777                                                                            

1142857 rows selected.


Execution Plan
----------------------------------------------------------                                                                                            
Plan hash value: 3956160932                                                                                                                           
                                                                                                                                                      
--------------------------------------------------------------------------                                                                            
| Id  | Operation         | Name | Rows  | Bytes | Cost (%CPU)| Time     |                                                                            
--------------------------------------------------------------------------                                                                            
|   0 | SELECT STATEMENT  |      |   400K|    12M|  1544  (10)| 00:00:01 |                                                                            
|*  1 |  TABLE ACCESS FULL| EMP  |   400K|    12M|  1544  (10)| 00:00:01 |                                                                            
--------------------------------------------------------------------------                                                                            
                                                                                                                                                      
Predicate Information (identified by operation id):                                                                                                   
---------------------------------------------------                                                                                                   
                                                                                                                                                      
   1 - filter("EMPSAL"=TO_NUMBER(:ESAL))                                                                                                              


Statistics
----------------------------------------------------------                                                                                            
          1  recursive calls                                                                                                                          
          0  db block gets                                                                                                                            
      86255  consistent gets                                                                                                                          
       4358  physical reads                                                                                                                           
          0  redo size                                                                                                                                
   54468801  bytes sent via SQL*Net to client                                                                                                         
     838641  bytes received via SQL*Net from client                                                                                                   
      76192  SQL*Net roundtrips to/from client                                                                                                        
          0  sorts (memory)                                                                                                                           
          0  sorts (disk)                                                                                                                             
    1142857  rows processed                                                                                                                           

SQL> 

The index access path no longer exists, even though the bind value of 1000 should generate that plan. Let’s reverse the bind variable assignment so Oracle looks for the records having a salary of 7000 first; the plans generated are, again, not what would be expected:


SQL> 
SQL> --
SQL> -- Create sample tables
SQL> --
SQL> create table emp(
  2  	     empid   number,
  3  	     empnm   varchar2(40),
  4  	     empsal  number,
  5  	     empssn  varchar2(12),
  6  	     constraint emp_pk primary key (empid)
  7  ) ;

Table created.

SQL> 
SQL> create index empsal_idx on emp(empsal) ;

Index created.

SQL> 
SQL> create table emp_dept(
  2  	     empid   number,
  3  	     empdept number,
  4  	     emploc  varchar2(60),
  5  	     constraint emp_dept_pk primary key(empid)
  6  ) ;

Table created.

SQL> 
SQL> create table dept_info(
  2  	     deptnum number,
  3  	     deptnm  varchar2(25),
  4  	     constraint dept_info_pk primary key(deptnum)
  5  ) ;

Table created.

SQL> 
SQL> --
SQL> -- Load sample tables with data
SQL> --
SQL> begin
  2  	     for i in 1..2000000 loop
  3  		     insert into emp
  4  		     values(i, 'Fnarm'||i, (mod(i, 7)+1)*1000, mod(i,10)||mod(i,10)||mod(i,10)||'-'||mod(i,10)||mod(i,10)||'-'||mod(i,10)||mod(i,10)||mod(i,10)||mod(i,10));
  5  		     insert into emp_dept
  6  		     values(i, (mod(i,8)+1)*10, 'Zanzwalla'||(mod(i,8)+1)*10);
  7  		     commit;
  8  	     end loop;
  9  	     update emp
 10  	     set empsal=7000 where empsal in (3000,4000,5000);
 11  	     update emp set empsal=800 where empsal=1000 and empssn not like '9%';
 12  	     commit;
 13  	     insert into dept_info
 14  	     select distinct empdept, case when empdept = 10 then 'SALES'
 15  					   when empdept = 20 then 'PROCUREMENT'
 16  					   when empdept = 30 then 'HR'
 17  					   when empdept = 40 then 'RESEARCH'
 18  					   when empdept = 50 then 'DEVELOPMENT'
 19  					   when empdept = 60 then 'EMPLOYEE RELATIONS'
 20  					   when empdept = 70 then 'FACILITIES'
 21  					   when empdept = 80 then 'FINANCE' end
 22  	     from emp_dept;
 23  
 24  end;
 25  /

PL/SQL procedure successfully completed.

SQL> exec dbms_stats.gather_schema_stats('BING')

PL/SQL procedure successfully completed.

SQL> 
SQL> --
SQL> -- Run query on indexed column
SQL> -- and do not use index
SQL> --
SQL> -- Generate execution plan
SQL> --
SQL> variable esal number
SQL> 
SQL> begin
  2  	     :esal:=7000;
  3  end;
  4  /

PL/SQL procedure successfully completed.

SQL> 
SQL> set autotrace on
SQL> 
SQL> select *
  2  from emp
  3  where empsal = :esal;

     EMPID EMPNM                                        EMPSAL EMPSSN                                                                                 
---------- ---------------------------------------- ---------- ------------                                                                           
       888 Fnarm888                                       7000 888-88-8888                                                                            
       891 Fnarm891                                       7000 111-11-1111                                                                            
       892 Fnarm892                                       7000 222-22-2222                                                                            
       893 Fnarm893                                       7000 333-33-3333                                                                            
...
   1937557 Fnarm1937557                                   7000 777-77-7777                                                                            

1142857 rows selected.


Execution Plan
----------------------------------------------------------                                                                                            
Plan hash value: 3956160932                                                                                                                           
                                                                                                                                                      
--------------------------------------------------------------------------                                                                            
| Id  | Operation         | Name | Rows  | Bytes | Cost (%CPU)| Time     |                                                                            
--------------------------------------------------------------------------                                                                            
|   0 | SELECT STATEMENT  |      |   400K|    12M|  1544  (10)| 00:00:01 |                                                                            
|*  1 |  TABLE ACCESS FULL| EMP  |   400K|    12M|  1544  (10)| 00:00:01 |                                                                            
--------------------------------------------------------------------------                                                                            
                                                                                                                                                      
Predicate Information (identified by operation id):                                                                                                   
---------------------------------------------------                                                                                                   
                                                                                                                                                      
   1 - filter("EMPSAL"=TO_NUMBER(:ESAL))                                                                                                              


Statistics
----------------------------------------------------------                                                                                            
         15  recursive calls                                                                                                                          
          0  db block gets                                                                                                                            
      96810  consistent gets                                                                                                                          
       4935  physical reads                                                                                                                           
     843308  redo size                                                                                                                                
   54468801  bytes sent via SQL*Net to client                                                                                                         
     838641  bytes received via SQL*Net from client                                                                                                   
      76192  SQL*Net roundtrips to/from client                                                                                                        
          0  sorts (memory)                                                                                                                           
          0  sorts (disk)                                                                                                                             
    1142857  rows processed                                                                                                                           

SQL> 
SQL> --
SQL> -- Now use indexed column and try to use index
SQL> --
SQL> begin
  2  	     :esal:=1000;
  3  end;
  4  /

PL/SQL procedure successfully completed.

SQL> 
SQL> select *
  2  from emp
  3  where empsal=:esal;

     EMPID EMPNM                                        EMPSAL EMPSSN                                                                                 
---------- ---------------------------------------- ---------- ------------                                                                           
       889 Fnarm889                                       1000 999-99-9999                                                                            
       959 Fnarm959                                       1000 999-99-9999                                                                            
      1029 Fnarm1029                                      1000 999-99-9999                                                                            
      1099 Fnarm1099                                      1000 999-99-9999                                                                            
...
   1934499 Fnarm1934499                                   1000 999-99-9999                                                                            
   1934569 Fnarm1934569                                   1000 999-99-9999                                                                            
   1937439 Fnarm1937439                                   1000 999-99-9999                                                                            
   1937509 Fnarm1937509                                   1000 999-99-9999                                                                            

28571 rows selected.


Execution Plan
----------------------------------------------------------                                                                                            
Plan hash value: 3956160932                                                                                                                           
                                                                                                                                                      
--------------------------------------------------------------------------                                                                            
| Id  | Operation         | Name | Rows  | Bytes | Cost (%CPU)| Time     |                                                                            
--------------------------------------------------------------------------                                                                            
|   0 | SELECT STATEMENT  |      |   400K|    12M|  1544  (10)| 00:00:01 |                                                                            
|*  1 |  TABLE ACCESS FULL| EMP  |   400K|    12M|  1544  (10)| 00:00:01 |                                                                            
--------------------------------------------------------------------------                                                                            
                                                                                                                                                      
Predicate Information (identified by operation id):                                                                                                   
---------------------------------------------------                                                                                                   
                                                                                                                                                      
   1 - filter("EMPSAL"=TO_NUMBER(:ESAL))                                                                                                              


Statistics
----------------------------------------------------------                                                                                            
          9  recursive calls                                                                                                                          
          0  db block gets                                                                                                                            
      16934  consistent gets                                                                                                                          
       5887  physical reads                                                                                                                           
      56732  redo size                                                                                                                                
    1454131  bytes sent via SQL*Net to client                                                                                                         
      21495  bytes received via SQL*Net from client                                                                                                   
       1906  SQL*Net roundtrips to/from client                                                                                                        
          0  sorts (memory)                                                                                                                           
          0  sorts (disk)                                                                                                                             
      28571  rows processed                                                                                                                           

SQL> 

It appears that regardless of the order Adaptive Cursor Sharing produces the same full table scan plan regardless of bind value, at least in 12.1.0.2. Statistics on the schema are current for both runs and using literal values in place of bind variables produces the expected execution plans so it appears that the statistics accurately reflect the data.

The person posting the query asked if this was a bug; if it’s related to histograms and cardinality then Bug 14565911 applies, and it would certainly seem to be the case given the evidence presented. Possibly putting more fuel on that fire let’s change how statistics are generated and see if that makes a difference:


SQL> 
SQL> --
SQL> -- Create sample tables
SQL> --
SQL> create table emp(
  2  	     empid   number,
  3  	     empnm   varchar2(40),
  4  	     empsal  number,
  5  	     empssn  varchar2(12),
  6  	     constraint emp_pk primary key (empid)
  7  ) ;

Table created.

SQL> 
SQL> create index empsal_idx on emp(empsal) ;

Index created.

SQL> 
SQL> create table emp_dept(
  2  	     empid   number,
  3  	     empdept number,
  4  	     emploc  varchar2(60),
  5  	     constraint emp_dept_pk primary key(empid)
  6  ) ;

Table created.

SQL> 
SQL> create table dept_info(
  2  	     deptnum number,
  3  	     deptnm  varchar2(25),
  4  	     constraint dept_info_pk primary key(deptnum)
  5  ) ;

Table created.

SQL> 
SQL> --
SQL> -- Load sample tables with data
SQL> --
SQL> begin
  2  	     for i in 1..2000000 loop
  3  		     insert into emp
  4  		     values(i, 'Fnarm'||i, (mod(i, 7)+1)*1000, mod(i,10)||mod(i,10)||mod(i,10)||'-'||mod(i,10)||mod(i,10)||'-'||mod(i,10)||mod(i,10)||mod(i,10)||mod(i,10));
  5  		     insert into emp_dept
  6  		     values(i, (mod(i,8)+1)*10, 'Zanzwalla'||(mod(i,8)+1)*10);
  7  		     commit;
  8  	     end loop;
  9  	     update emp
 10  	     set empsal=7000 where empsal in (3000,4000,5000);
 11  	     update emp set empsal=800 where empsal=1000 and empssn not like '9%';
 12  	     commit;
 13  	     insert into dept_info
 14  	     select distinct empdept, case when empdept = 10 then 'SALES'
 15  					   when empdept = 20 then 'PROCUREMENT'
 16  					   when empdept = 30 then 'HR'
 17  					   when empdept = 40 then 'RESEARCH'
 18  					   when empdept = 50 then 'DEVELOPMENT'
 19  					   when empdept = 60 then 'EMPLOYEE RELATIONS'
 20  					   when empdept = 70 then 'FACILITIES'
 21  					   when empdept = 80 then 'FINANCE' end
 22  	     from emp_dept;
 23  
 24  end;
 25  /

PL/SQL procedure successfully completed.

SQL> 
SQL> exec dbms_stats.gather_schema_stats('BING', method_opt => 'for all columns size skewonly')

PL/SQL procedure successfully completed.

SQL> 
SQL> --
SQL> -- Run query on indexed column
SQL> -- and do not use index
SQL> --
SQL> -- Generate execution plan
SQL> --
SQL> variable esal number
SQL> 
SQL> begin
  2  	     :esal:=7000;
  3  end;
  4  /

PL/SQL procedure successfully completed.

SQL> 
SQL> set autotrace on
SQL> 
SQL> select *
  2  from emp
  3  where empsal = :esal;

     EMPID EMPNM                                        EMPSAL EMPSSN                                                                                 
---------- ---------------------------------------- ---------- ------------                                                                           
       888 Fnarm888                                       7000 888-88-8888                                                                            
       891 Fnarm891                                       7000 111-11-1111                                                                            
       892 Fnarm892                                       7000 222-22-2222                                                                            
       893 Fnarm893                                       7000 333-33-3333                                                                            
...
   1955225 Fnarm1955225                                   7000 555-55-5555                                                                            

1142857 rows selected.


Execution Plan
----------------------------------------------------------                                                                                            
Plan hash value: 3956160932                                                                                                                           
                                                                                                                                                      
--------------------------------------------------------------------------                                                                            
| Id  | Operation         | Name | Rows  | Bytes | Cost (%CPU)| Time     |                                                                            
--------------------------------------------------------------------------                                                                            
|   0 | SELECT STATEMENT  |      |   400K|    12M|  1544  (10)| 00:00:01 |                                                                            
|*  1 |  TABLE ACCESS FULL| EMP  |   400K|    12M|  1544  (10)| 00:00:01 |                                                                            
--------------------------------------------------------------------------                                                                            
                                                                                                                                                      
Predicate Information (identified by operation id):                                                                                                   
---------------------------------------------------                                                                                                   
                                                                                                                                                      
   1 - filter("EMPSAL"=TO_NUMBER(:ESAL))                                                                                                              


Statistics
----------------------------------------------------------                                                                                            
          1  recursive calls                                                                                                                          
          0  db block gets                                                                                                                            
      86250  consistent gets                                                                                                                          
       7494  physical reads                                                                                                                           
          0  redo size                                                                                                                                
   54469617  bytes sent via SQL*Net to client                                                                                                         
     838641  bytes received via SQL*Net from client                                                                                                   
      76192  SQL*Net roundtrips to/from client                                                                                                        
          0  sorts (memory)                                                                                                                           
          0  sorts (disk)                                                                                                                             
    1142857  rows processed                                                                                                                           

SQL> 
SQL> --
SQL> -- Now use indexed column and try to use index
SQL> --
SQL> begin
  2  	     :esal:=1000;
  3  end;
  4  /

PL/SQL procedure successfully completed.

SQL> 
SQL> select *
  2  from emp
  3  where empsal=:esal;

     EMPID EMPNM                                        EMPSAL EMPSSN                                                                                 
---------- ---------------------------------------- ---------- ------------                                                                           
       889 Fnarm889                                       1000 999-99-9999                                                                            
       959 Fnarm959                                       1000 999-99-9999                                                                            
      1029 Fnarm1029                                      1000 999-99-9999                                                                            
      1099 Fnarm1099                                      1000 999-99-9999                                                                            
...
   1952209 Fnarm1952209                                   1000 999-99-9999                                                                            
   1955079 Fnarm1955079                                   1000 999-99-9999                                                                            
   1955149 Fnarm1955149                                   1000 999-99-9999                                                                            
   1955219 Fnarm1955219                                   1000 999-99-9999                                                                            

28571 rows selected.


Execution Plan
----------------------------------------------------------                                                                                            
Plan hash value: 3956160932                                                                                                                           
                                                                                                                                                      
--------------------------------------------------------------------------                                                                            
| Id  | Operation         | Name | Rows  | Bytes | Cost (%CPU)| Time     |                                                                            
--------------------------------------------------------------------------                                                                            
|   0 | SELECT STATEMENT  |      |   400K|    12M|  1544  (10)| 00:00:01 |                                                                            
|*  1 |  TABLE ACCESS FULL| EMP  |   400K|    12M|  1544  (10)| 00:00:01 |                                                                            
--------------------------------------------------------------------------                                                                            
                                                                                                                                                      
Predicate Information (identified by operation id):                                                                                                   
---------------------------------------------------                                                                                                   
                                                                                                                                                      
   1 - filter("EMPSAL"=TO_NUMBER(:ESAL))                                                                                                              


Statistics
----------------------------------------------------------                                                                                            
          1  recursive calls                                                                                                                          
          0  db block gets                                                                                                                            
      16705  consistent gets                                                                                                                          
       7930  physical reads                                                                                                                           
          0  redo size                                                                                                                                
    1454131  bytes sent via SQL*Net to client                                                                                                         
      21495  bytes received via SQL*Net from client                                                                                                   
       1906  SQL*Net roundtrips to/from client                                                                                                        
          0  sorts (memory)                                                                                                                           
          0  sorts (disk)                                                                                                                             
      28571  rows processed                                                                                                                           

SQL> 

No change even when histograms are generated only for skewed column data. So this does seem to be affected by Bug 14565911, and no patch appears to be available. The bug is listed as fixed in release 12.2; it may be necessary to get a backport of this bug fix through My Oracle Support.

With a product as complex as Oracle bugs and issues are bound to turn up; not every case can be tested prior to release. As the bug report states this is restricted to bind variable usage. Notice that for all of the execution plans the predicate information reports that the bind variable, although declared as a number and populated as such, is being processed by the TO_NUMBER function, which may influence the index usage. Let’s give this one more try, this time with a function-based index using the TO_NUMBER function:


SQL> 
SQL> --
SQL> -- Create sample tables
SQL> --
SQL> create table emp(
  2  	     empid   number,
  3  	     empnm   varchar2(40),
  4  	     empsal  number,
  5  	     empssn  varchar2(12),
  6  	     constraint emp_pk primary key (empid)
  7  ) ;

Table created.

SQL> 
SQL> create index empsal_idx on emp(empsal) ;

Index created.

SQL> create index empsal_fbi_idx on emp(to_number(empsal)) ;

Index created.

SQL> 
SQL> create table emp_dept(
  2  	     empid   number,
  3  	     empdept number,
  4  	     emploc  varchar2(60),
  5  	     constraint emp_dept_pk primary key(empid)
  6  ) ;

Table created.

SQL> 
SQL> create table dept_info(
  2  	     deptnum number,
  3  	     deptnm  varchar2(25),
  4  	     constraint dept_info_pk primary key(deptnum)
  5  ) ;

Table created.

SQL> 
SQL> --
SQL> -- Load sample tables with data
SQL> --
SQL> begin
  2  	     for i in 1..2000000 loop
  3  		     insert into emp
  4  		     values(i, 'Fnarm'||i, (mod(i, 7)+1)*1000, mod(i,10)||mod(i,10)||mod(i,10)||'-'||mod(i,10)||mod(i,10)||'-'||mod(i,10)||mod(i,10)||mod(i,10)||mod(i,10));
  5  		     insert into emp_dept
  6  		     values(i, (mod(i,8)+1)*10, 'Zanzwalla'||(mod(i,8)+1)*10);
  7  		     commit;
  8  	     end loop;
  9  	     update emp
 10  	     set empsal=7000 where empsal in (3000,4000,5000);
 11  	     commit;
 12  	     update emp set empsal=800 where empsal=1000 and empssn not like '9%';
 13  	     commit;
 14  	     insert into dept_info
 15  	     select distinct empdept, case when empdept = 10 then 'SALES'
 16  					   when empdept = 20 then 'PROCUREMENT'
 17  					   when empdept = 30 then 'HR'
 18  					   when empdept = 40 then 'RESEARCH'
 19  					   when empdept = 50 then 'DEVELOPMENT'
 20  					   when empdept = 60 then 'EMPLOYEE RELATIONS'
 21  					   when empdept = 70 then 'FACILITIES'
 22  					   when empdept = 80 then 'FINANCE' end
 23  	     from emp_dept;
 24  
 25  end;
 26  /

PL/SQL procedure successfully completed.

SQL> 
SQL> exec dbms_stats.gather_schema_stats('BING')

PL/SQL procedure successfully completed.

SQL> 
SQL> --
SQL> -- Run query on indexed column
SQL> -- and use index
SQL> --
SQL> -- Generate execution plan
SQL> --
SQL> variable esal number
SQL> 
SQL> begin
  2  	     :esal:=1000;
  3  end;
  4  /

PL/SQL procedure successfully completed.

SQL> 
SQL> set autotrace on
SQL> 
SQL> select *
  2  from emp
  3  where empsal = :esal;

     EMPID EMPNM                                        EMPSAL EMPSSN                                                                                 
---------- ---------------------------------------- ---------- ------------                                                                           
       679 Fnarm679                                       1000 999-99-9999                                                                            
       749 Fnarm749                                       1000 999-99-9999                                                                            
       819 Fnarm819                                       1000 999-99-9999                                                                            
       889 Fnarm889                                       1000 999-99-9999                                                                            
       959 Fnarm959                                       1000 999-99-9999                                                                            
...
   1451849 Fnarm1451849                                   1000 999-99-9999                                                                            
   1451219 Fnarm1451219                                   1000 999-99-9999                                                                            
   1451289 Fnarm1451289                                   1000 999-99-9999                                                                            
   1451779 Fnarm1451779                                   1000 999-99-9999                                                                            

28571 rows selected.


Execution Plan
----------------------------------------------------------                                                                                            
Plan hash value: 3956160932                                                                                                                           
                                                                                                                                                      
--------------------------------------------------------------------------                                                                            
| Id  | Operation         | Name | Rows  | Bytes | Cost (%CPU)| Time     |                                                                            
--------------------------------------------------------------------------                                                                            
|   0 | SELECT STATEMENT  |      |   400K|    12M|  1544  (10)| 00:00:01 |                                                                            
|*  1 |  TABLE ACCESS FULL| EMP  |   400K|    12M|  1544  (10)| 00:00:01 |                                                                            
--------------------------------------------------------------------------                                                                            
                                                                                                                                                      
Predicate Information (identified by operation id):                                                                                                   
---------------------------------------------------                                                                                                   
                                                                                                                                                      
   1 - filter("EMPSAL"=TO_NUMBER(:ESAL))                                                                                                              


Statistics
----------------------------------------------------------                                                                                            
          1  recursive calls                                                                                                                          
          0  db block gets                                                                                                                            
      16774  consistent gets                                                                                                                          
       8138  physical reads                                                                                                                           
          0  redo size                                                                                                                                
    1454131  bytes sent via SQL*Net to client                                                                                                         
      21496  bytes received via SQL*Net from client                                                                                                   
       1906  SQL*Net roundtrips to/from client                                                                                                        
          0  sorts (memory)                                                                                                                           
          0  sorts (disk)                                                                                                                             
      28571  rows processed                                                                                                                           

SQL> 
SQL> --
SQL> -- Now use indexed column and NOT use index
SQL> --
SQL> begin
  2  	     :esal:=7000;
  3  end;
  4  /

PL/SQL procedure successfully completed.

SQL> 
SQL> select *
  2  from emp
  3  where empsal=:esal;

     EMPID EMPNM                                        EMPSAL EMPSSN                                                                                 
---------- ---------------------------------------- ---------- ------------                                                                           
       667 Fnarm667                                       7000 777-77-7777                                                                            
       668 Fnarm668                                       7000 888-88-8888                                                                            
       669 Fnarm669                                       7000 999-99-9999                                                                            
...
   1934580 Fnarm1934580                                   7000 000-00-0000                                                                            

1142857 rows selected.


Execution Plan
----------------------------------------------------------                                                                                            
Plan hash value: 3956160932                                                                                                                           
                                                                                                                                                      
--------------------------------------------------------------------------                                                                            
| Id  | Operation         | Name | Rows  | Bytes | Cost (%CPU)| Time     |                                                                            
--------------------------------------------------------------------------                                                                            
|   0 | SELECT STATEMENT  |      |   400K|    12M|  1544  (10)| 00:00:01 |                                                                            
|*  1 |  TABLE ACCESS FULL| EMP  |   400K|    12M|  1544  (10)| 00:00:01 |                                                                            
--------------------------------------------------------------------------                                                                            
                                                                                                                                                      
Predicate Information (identified by operation id):                                                                                                   
---------------------------------------------------                                                                                                   
                                                                                                                                                      
   1 - filter("EMPSAL"=TO_NUMBER(:ESAL))                                                                                                              


Statistics
----------------------------------------------------------                                                                                            
          1  recursive calls                                                                                                                          
          0  db block gets                                                                                                                            
      86316  consistent gets                                                                                                                          
       7524  physical reads                                                                                                                           
          0  redo size                                                                                                                                
   54462057  bytes sent via SQL*Net to client                                                                                                         
     838642  bytes received via SQL*Net from client                                                                                                   
      76192  SQL*Net roundtrips to/from client                                                                                                        
          0  sorts (memory)                                                                                                                           
          0  sorts (disk)                                                                                                                             
    1142857  rows processed                                                                                                                           

SQL> 

The function-based index made no difference, and it really shouldn’t have been expected to. The test was included to provide most of the possible scenarios for plan verification, and it seems that the bug isn’t mitigated by how the stats are gathered or which indexes are created. [Not every possible combination of calls to DBMS_STATS were tried as doing that could take several hours to generate and run every test case.] In 12.1.0.2 you could run into a situation where bad plans are being generated when bind variables are used. Knowing this may help explain some cases of poor performance where bind variables are used.

Let’s see if dynamic sampling produces different results:


SQL> 
SQL> --
SQL> -- Create sample tables
SQL> --
SQL> -- Create them parallel, necessary
SQL> -- to get a Smart Scan on these tables
SQL> --
SQL> create table emp(
  2  	     empid   number,
  3  	     empnm   varchar2(40),
  4  	     empsal  number,
  5  	     empssn  varchar2(12),
  6  	     constraint emp_pk primary key (empid)
  7  ) ;

Table created.

SQL> 
SQL> create index empsal_idx on emp(empsal) ;

Index created.

SQL> 
SQL> create table emp_dept(
  2  	     empid   number,
  3  	     empdept number,
  4  	     emploc  varchar2(60),
  5  	     constraint emp_dept_pk primary key(empid)
  6  ) ;

Table created.

SQL> 
SQL> create table dept_info(
  2  	     deptnum number,
  3  	     deptnm  varchar2(25),
  4  	     constraint dept_info_pk primary key(deptnum)
  5  ) ;

Table created.

SQL> 
SQL> --
SQL> -- Load sample tables with data
SQL> --
SQL> begin
  2  	     for i in 1..2000000 loop
  3  		     insert into emp
  4  		     values(i, 'Fnarm'||i, (mod(i, 7)+1)*1000, mod(i,10)||mod(i,10)||mod(i,10)||'-'||mod(i,10)||mod(i,10)||'-'||mod(i,10)||mod(i,10)||mod(i,10)||mod(i,10));
  5  		     insert into emp_dept
  6  		     values(i, (mod(i,8)+1)*10, 'Zanzwalla'||(mod(i,8)+1)*10);
  7  		     commit;
  8  	     end loop;
  9  	     update emp
 10  	     set empsal=7000 where empsal in (3000,4000,5000);
 11  	     commit;
 12  	     update emp set empsal=800 where empsal=1000 and empssn not like '9%';
 13  	     commit;
 14  	     insert into dept_info
 15  	     select distinct empdept, case when empdept = 10 then 'SALES'
 16  					   when empdept = 20 then 'PROCUREMENT'
 17  					   when empdept = 30 then 'HR'
 18  					   when empdept = 40 then 'RESEARCH'
 19  					   when empdept = 50 then 'DEVELOPMENT'
 20  					   when empdept = 60 then 'EMPLOYEE RELATIONS'
 21  					   when empdept = 70 then 'FACILITIES'
 22  					   when empdept = 80 then 'FINANCE' end
 23  	     from emp_dept;
 24  
 25  end;
 26  /

PL/SQL procedure successfully completed.

SQL> 
SQL> --
SQL> -- Run query on indexed column
SQL> -- and use index
SQL> --
SQL> -- Generate execution plan
SQL> --
SQL> -- Also report query execution time
SQL> --
SQL> variable esal number
SQL> 
SQL> begin
  2  	     :esal:=1000;
  3  end;
  4  /

PL/SQL procedure successfully completed.

SQL> 
SQL> set autotrace on
SQL> 
SQL> select *
  2  from emp
  3  where empsal = :esal;

     EMPID EMPNM                                        EMPSAL EMPSSN                                                                                 
---------- ---------------------------------------- ---------- ------------                                                                           
       679 Fnarm679                                       1000 999-99-9999                                                                            
       749 Fnarm749                                       1000 999-99-9999                                                                            
       819 Fnarm819                                       1000 999-99-9999                                                                            
...
   1973629 Fnarm1973629                                   1000 999-99-9999                                                                            
   1934429 Fnarm1934429                                   1000 999-99-9999                                                                            
   1934499 Fnarm1934499                                   1000 999-99-9999                                                                            
   1934569 Fnarm1934569                                   1000 999-99-9999                                                                            

28571 rows selected.


Execution Plan
----------------------------------------------------------                                                                                            
Plan hash value: 3668658578                                                                                                                           
                                                                                                                                                      
--------------------------------------------------------------------------------------------------                                                    
| Id  | Operation                           | Name       | Rows  | Bytes | Cost (%CPU)| Time     |                                                    
--------------------------------------------------------------------------------------------------                                                    
|   0 | SELECT STATEMENT                    |            | 15472 |   846K|     1   (0)| 00:00:01 |                                                    
|   1 |  TABLE ACCESS BY INDEX ROWID BATCHED| EMP        | 15472 |   846K|     1   (0)| 00:00:01 |                                                    
|*  2 |   INDEX RANGE SCAN                  | EMPSAL_IDX |  6189 |       |     1   (0)| 00:00:01 |                                                    
--------------------------------------------------------------------------------------------------                                                    
                                                                                                                                                      
Predicate Information (identified by operation id):                                                                                                   
---------------------------------------------------                                                                                                   
                                                                                                                                                      
   2 - access("EMPSAL"=TO_NUMBER(:ESAL))                                                                                                              
                                                                                                                                                      
Note                                                                                                                                                  
-----                                                                                                                                                 
   - dynamic statistics used: dynamic sampling (level=2)                                                                                              


Statistics
----------------------------------------------------------                                                                                            
         17  recursive calls                                                                                                                          
          0  db block gets                                                                                                                            
      27718  consistent gets                                                                                                                          
       6720  physical reads                                                                                                                           
     901140  redo size                                                                                                                                
    1454131  bytes sent via SQL*Net to client                                                                                                         
      21496  bytes received via SQL*Net from client                                                                                                   
       1906  SQL*Net roundtrips to/from client                                                                                                        
          0  sorts (memory)                                                                                                                           
          0  sorts (disk)                                                                                                                             
      28571  rows processed                                                                                                                           

SQL> 
SQL> --
SQL> -- Now use indexed column and NOT use index
SQL> --
SQL> begin
  2  	     :esal:=7000;
  3  end;
  4  /

PL/SQL procedure successfully completed.

SQL> 
SQL> select *
  2  from emp
  3  where empsal=:esal;

     EMPID EMPNM                                        EMPSAL EMPSSN                                                                                 
---------- ---------------------------------------- ---------- ------------                                                                           
       667 Fnarm667                                       7000 777-77-7777                                                                            
       668 Fnarm668                                       7000 888-88-8888                                                                            
       669 Fnarm669                                       7000 999-99-9999                                                                            
...
   1934580 Fnarm1934580                                   7000 000-00-0000                                                                            

1142857 rows selected.


Execution Plan
----------------------------------------------------------                                                                                            
Plan hash value: 3668658578                                                                                                                           
                                                                                                                                                      
--------------------------------------------------------------------------------------------------                                                    
| Id  | Operation                           | Name       | Rows  | Bytes | Cost (%CPU)| Time     |                                                    
--------------------------------------------------------------------------------------------------                                                    
|   0 | SELECT STATEMENT                    |            | 15472 |   846K|     1   (0)| 00:00:01 |                                                    
|   1 |  TABLE ACCESS BY INDEX ROWID BATCHED| EMP        | 15472 |   846K|     1   (0)| 00:00:01 |                                                    
|*  2 |   INDEX RANGE SCAN                  | EMPSAL_IDX |  6189 |       |     1   (0)| 00:00:01 |                                                    
--------------------------------------------------------------------------------------------------                                                    
                                                                                                                                                      
Predicate Information (identified by operation id):                                                                                                   
---------------------------------------------------                                                                                                   
                                                                                                                                                      
   2 - access("EMPSAL"=TO_NUMBER(:ESAL))                                                                                                              
                                                                                                                                                      
Note                                                                                                                                                  
-----                                                                                                                                                 
   - dynamic statistics used: dynamic sampling (level=2)                                                                                              


Statistics
----------------------------------------------------------                                                                                            
         10  recursive calls                                                                                                                          
          0  db block gets                                                                                                                            
      86407  consistent gets                                                                                                                          
       7293  physical reads                                                                                                                           
        856  redo size                                                                                                                                
   54462057  bytes sent via SQL*Net to client                                                                                                         
     838642  bytes received via SQL*Net from client                                                                                                   
      76192  SQL*Net roundtrips to/from client                                                                                                        
          0  sorts (memory)                                                                                                                           
          0  sorts (disk)                                                                                                                             
    1142857  rows processed                                                                                                                           

SQL> 

Doesn’t look good; maybe if the conditions are reversed:


SQL> 
SQL> --
SQL> -- Create sample tables
SQL> --
SQL> -- Create them parallel, necessary
SQL> -- to get a Smart Scan on these tables
SQL> --
SQL> create table emp(
  2  	     empid   number,
  3  	     empnm   varchar2(40),
  4  	     empsal  number,
  5  	     empssn  varchar2(12),
  6  	     constraint emp_pk primary key (empid)
  7  ) ;

Table created.

SQL> 
SQL> create index empsal_idx on emp(empsal) ;

Index created.

SQL> 
SQL> create table emp_dept(
  2  	     empid   number,
  3  	     empdept number,
  4  	     emploc  varchar2(60),
  5  	     constraint emp_dept_pk primary key(empid)
  6  ) ;

Table created.

SQL> 
SQL> create table dept_info(
  2  	     deptnum number,
  3  	     deptnm  varchar2(25),
  4  	     constraint dept_info_pk primary key(deptnum)
  5  ) ;

Table created.

SQL> 
SQL> --
SQL> -- Load sample tables with data
SQL> --
SQL> begin
  2  	     for i in 1..2000000 loop
  3  		     insert into emp
  4  		     values(i, 'Fnarm'||i, (mod(i, 7)+1)*1000, mod(i,10)||mod(i,10)||mod(i,10)||'-'||mod(i,10)||mod(i,10)||'-'||mod(i,10)||mod(i,10)||mod(i,10)||mod(i,10));
  5  		     insert into emp_dept
  6  		     values(i, (mod(i,8)+1)*10, 'Zanzwalla'||(mod(i,8)+1)*10);
  7  		     commit;
  8  	     end loop;
  9  	     update emp
 10  	     set empsal=7000 where empsal in (3000,4000,5000);
 11  	     commit;
 12  	     update emp set empsal=800 where empsal=1000 and empssn not like '9%';
 13  	     commit;
 14  	     insert into dept_info
 15  	     select distinct empdept, case when empdept = 10 then 'SALES'
 16  					   when empdept = 20 then 'PROCUREMENT'
 17  					   when empdept = 30 then 'HR'
 18  					   when empdept = 40 then 'RESEARCH'
 19  					   when empdept = 50 then 'DEVELOPMENT'
 20  					   when empdept = 60 then 'EMPLOYEE RELATIONS'
 21  					   when empdept = 70 then 'FACILITIES'
 22  					   when empdept = 80 then 'FINANCE' end
 23  	     from emp_dept;
 24  
 25  end;
 26  /

PL/SQL procedure successfully completed.

SQL> 
SQL> --
SQL> -- Run query on indexed column
SQL> -- and not use index
SQL> --
SQL> -- Generate execution plan
SQL> --
SQL> -- Also report query execution time
SQL> --
SQL> variable esal number
SQL> 
SQL> begin
  2  	     :esal:=7000;
  3  end;
  4  /

PL/SQL procedure successfully completed.

SQL> 
SQL> set autotrace on
SQL> 
SQL> select *
  2  from emp
  3  where empsal = :esal;

     EMPID EMPNM                                        EMPSAL EMPSSN                                                                                 
---------- ---------------------------------------- ---------- ------------                                                                           
       888 Fnarm888                                       7000 888-88-8888                                                                            
       891 Fnarm891                                       7000 111-11-1111                                                                            
       892 Fnarm892                                       7000 222-22-2222                                                                            
...
   1937557 Fnarm1937557                                   7000 777-77-7777                                                                            

1142857 rows selected.


Execution Plan
----------------------------------------------------------                                                                                            
Plan hash value: 3668658578                                                                                                                           
                                                                                                                                                      
--------------------------------------------------------------------------------------------------                                                    
| Id  | Operation                           | Name       | Rows  | Bytes | Cost (%CPU)| Time     |                                                    
--------------------------------------------------------------------------------------------------                                                    
|   0 | SELECT STATEMENT                    |            | 15237 |   833K|     1   (0)| 00:00:01 |                                                    
|   1 |  TABLE ACCESS BY INDEX ROWID BATCHED| EMP        | 15237 |   833K|     1   (0)| 00:00:01 |                                                    
|*  2 |   INDEX RANGE SCAN                  | EMPSAL_IDX |  6095 |       |     1   (0)| 00:00:01 |                                                    
--------------------------------------------------------------------------------------------------                                                    
                                                                                                                                                      
Predicate Information (identified by operation id):                                                                                                   
---------------------------------------------------                                                                                                   
                                                                                                                                                      
   2 - access("EMPSAL"=TO_NUMBER(:ESAL))                                                                                                              
                                                                                                                                                      
Note                                                                                                                                                  
-----                                                                                                                                                 
   - dynamic statistics used: dynamic sampling (level=2)                                                                                              


Statistics
----------------------------------------------------------                                                                                            
         16  recursive calls                                                                                                                          
          0  db block gets                                                                                                                            
      96812  consistent gets                                                                                                                          
       6017  physical reads                                                                                                                           
     841416  redo size                                                                                                                                
   54468801  bytes sent via SQL*Net to client                                                                                                         
     838641  bytes received via SQL*Net from client                                                                                                   
      76192  SQL*Net roundtrips to/from client                                                                                                        
          0  sorts (memory)                                                                                                                           
          0  sorts (disk)                                                                                                                             
    1142857  rows processed                                                                                                                           

SQL> 
SQL> --
SQL> -- Now use indexed column and use index
SQL> --
SQL> begin
  2  	     :esal:=1000;
  3  end;
  4  /

PL/SQL procedure successfully completed.

SQL> 
SQL> select *
  2  from emp
  3  where empsal=:esal;

     EMPID EMPNM                                        EMPSAL EMPSSN                                                                                 
---------- ---------------------------------------- ---------- ------------                                                                           
       889 Fnarm889                                       1000 999-99-9999                                                                            
       959 Fnarm959                                       1000 999-99-9999                                                                            
      1029 Fnarm1029                                      1000 999-99-9999                                                                            
...
   1934499 Fnarm1934499                                   1000 999-99-9999                                                                            
   1934569 Fnarm1934569                                   1000 999-99-9999                                                                            
   1937439 Fnarm1937439                                   1000 999-99-9999                                                                            
   1937509 Fnarm1937509                                   1000 999-99-9999                                                                            

28571 rows selected.


Execution Plan
----------------------------------------------------------                                                                                            
Plan hash value: 3668658578                                                                                                                           
                                                                                                                                                      
--------------------------------------------------------------------------------------------------                                                    
| Id  | Operation                           | Name       | Rows  | Bytes | Cost (%CPU)| Time     |                                                    
--------------------------------------------------------------------------------------------------                                                    
|   0 | SELECT STATEMENT                    |            | 15237 |   833K|     1   (0)| 00:00:01 |                                                    
|   1 |  TABLE ACCESS BY INDEX ROWID BATCHED| EMP        | 15237 |   833K|     1   (0)| 00:00:01 |                                                    
|*  2 |   INDEX RANGE SCAN                  | EMPSAL_IDX |  6095 |       |     1   (0)| 00:00:01 |                                                    
--------------------------------------------------------------------------------------------------                                                    
                                                                                                                                                      
Predicate Information (identified by operation id):                                                                                                   
---------------------------------------------------                                                                                                   
                                                                                                                                                      
   2 - access("EMPSAL"=TO_NUMBER(:ESAL))                                                                                                              
                                                                                                                                                      
Note                                                                                                                                                  
-----                                                                                                                                                 
   - dynamic statistics used: dynamic sampling (level=2)                                                                                              


Statistics
----------------------------------------------------------                                                                                            
         85  recursive calls                                                                                                                          
          0  db block gets                                                                                                                            
      17374  consistent gets                                                                                                                          
       7311  physical reads                                                                                                                           
      56732  redo size                                                                                                                                
    1454131  bytes sent via SQL*Net to client                                                                                                         
      21495  bytes received via SQL*Net from client                                                                                                   
       1906  SQL*Net roundtrips to/from client                                                                                                        
          0  sorts (memory)                                                                                                                           
          0  sorts (disk)                                                                                                                             
      28571  rows processed                                                                                                                           

SQL> 

This is even worse, as no FULL TABLE SCAN plan is used, even when it would be the most efficient.

Adaptive Cursor Sharing has been very good at producing good execution plans when bind variables are used with varying values; in 12.1.0.2 this has been adversely affected by Bug 14565911. Hopefully Oracle will issue a patch to correct this in releases from 11.1 on so we won’t have to wait for 12.2 to provide the expected behavior for Adaptive Cursor Sharing.

Seems mice and men do better, sometimes.

December 8, 2014

You Can Get There Making All Right-Hand Turns But …

Filed under: General — dfitzjarrell @ 08:12

It would appear that some DBAs are still using the optimizer_index_cost_adj parameter to make index access paths more ‘desirable’ to the optimizer. In decades past this might have been a good strategy however with the improvement in statistics gathering in recent relesaes of Oracle this might not be the case. Let’s look at an example to see why this might do more ‘harm’ than good.

The optimizer_index_cost_adj parameter was first provided in Oracle 9i as a way to ‘gently’ influence the Cost-Based Optimizer to favor index scans over full table scans. It did that rather efficiently and it still does, which brings us to the inherent problem of using it: it does its job all TOO well sometimes. For efficiency and reduced physical I/O sometimes a full table scan is better than using an index, a fact that’s been known in Oracle circles for years now. Still, some sites are still setting optimizer_index_cost_adj to a non-default value, possibly through migrations to newer relesaeas and failing to modify the init.ora file configured for the older version. Using 11.2.0.4 here’s an example of what can occur when this parameter is altered to what might seem to be a ‘reasonable’ value. We start by disabling the automatic statistics gathering with the CREATE INDEX statement:


SQL> 
SQL> alter session set "_optimizer_compute_index_stats"=false;

Session altered.

SQL> 

Now, create the index on the EMP table:


SQL> create index emp_idx on emp(job);

Index created.

Let’s set optimizer_index_cost_adj to 10 and see what plans we get for two queries, one that should generate a full table scan and one that should use the index:


SQL> 
SQL> set autotrace on
SQL> 
SQL> alter session set optimizer_index_cost_adj=10;

Session altered.

SQL> 
SQL> select empno, ename, job, sal
  2  from emp
  3  where job = 'CLERK';

     EMPNO ENAME      JOB              SAL                                                                                          
---------- ---------- --------- ----------                                                                                          
      7369 SMITH      CLERK            800                                                                                          
      7876 ADAMS      CLERK           1100                                                                                          
      7900 JAMES      CLERK            950                                                                                          
      7934 MILLER     CLERK           1300                                                                                          
      7369 SMITH      CLERK            800                                                                                          
...
      7934 MILLER     CLERK           1300                                                                                          

1024 rows selected.


Execution Plan
----------------------------------------------------------                                                                          
Plan hash value: 1472992808                                                                                                         
                                                                                                                                    
---------------------------------------------------------------------------------------                                             
| Id  | Operation                   | Name    | Rows  | Bytes | Cost (%CPU)| Time     |                                             
---------------------------------------------------------------------------------------                                             
|   0 | SELECT STATEMENT            |         |  1024 | 39936 |     6   (0)| 00:00:01 |                                             
|   1 |  TABLE ACCESS BY INDEX ROWID| EMP     |  1024 | 39936 |     6   (0)| 00:00:01 |                                             
|*  2 |   INDEX RANGE SCAN          | EMP_IDX |  1024 |       |     1   (0)| 00:00:01 |                                             
---------------------------------------------------------------------------------------                                             
                                                                                                                                    
Predicate Information (identified by operation id):                                                                                 
---------------------------------------------------                                                                                 
                                                                                                                                    
   2 - access("JOB"='CLERK')                                                                                                        
                                                                                                                                    
Note                                                                                                                                
-----                                                                                                                               
   - dynamic sampling used for this statement (level=2)                                                                             


Statistics
----------------------------------------------------------                                                                          
          9  recursive calls                                                                                                        
          0  db block gets                                                                                                          
        204  consistent gets                                                                                                        
          4  physical reads                                                                                                         
          0  redo size                                                                                                              
      31902  bytes sent via SQL*Net to client                                                                                       
       1247  bytes received via SQL*Net from client                                                                                 
         70  SQL*Net roundtrips to/from client                                                                                      
          0  sorts (memory)                                                                                                         
          0  sorts (disk)                                                                                                           
       1024  rows processed                                                                                                         

SQL> 
SQL> select empno, ename, job, sal
  2  from emp
  3  where job = 'PRESIDENT';

     EMPNO ENAME      JOB              SAL                                                                                          
---------- ---------- --------- ----------                                                                                          
      7839 KING       PRESIDENT       5000                                                                                          
      7869 JACK       PRESIDENT       5000                                                                                          


Execution Plan
----------------------------------------------------------                                                                          
Plan hash value: 1472992808                                                                                                         
                                                                                                                                    
---------------------------------------------------------------------------------------                                             
| Id  | Operation                   | Name    | Rows  | Bytes | Cost (%CPU)| Time     |                                             
---------------------------------------------------------------------------------------                                             
|   0 | SELECT STATEMENT            |         |     2 |    78 |     1   (0)| 00:00:01 |                                             
|   1 |  TABLE ACCESS BY INDEX ROWID| EMP     |     2 |    78 |     1   (0)| 00:00:01 |                                             
|*  2 |   INDEX RANGE SCAN          | EMP_IDX |     2 |       |     1   (0)| 00:00:01 |                                             
---------------------------------------------------------------------------------------                                             
                                                                                                                                    
Predicate Information (identified by operation id):                                                                                 
---------------------------------------------------                                                                                 
                                                                                                                                    
   2 - access("JOB"='PRESIDENT')                                                                                                    
                                                                                                                                    
Note                                                                                                                                
-----                                                                                                                               
   - dynamic sampling used for this statement (level=2)                                                                             


Statistics
----------------------------------------------------------                                                                          
          8  recursive calls                                                                                                        
          0  db block gets                                                                                                          
         40  consistent gets                                                                                                        
          1  physical reads                                                                                                         
          0  redo size                                                                                                              
        812  bytes sent via SQL*Net to client                                                                                       
        499  bytes received via SQL*Net from client                                                                                 
          2  SQL*Net roundtrips to/from client                                                                                      
          0  sorts (memory)                                                                                                         
          0  sorts (disk)                                                                                                           
          2  rows processed                                                                                                         

SQL> 

Notice that both queries used the index. This is because the index cost was artificially lowered by the optimizer_index_cost_adj setting in force. This was a fairly small table (4098 rows) and that 25% of the total row count was returned by the first query. Let’s now adjust the parameter setting back to its default and generate statistics on the schema:


SQL> alter session set optimizer_index_cost_adj=100;

Session altered.

SQL> 
SQL> exec dbms_stats.gather_schema_stats('BING')

PL/SQL procedure successfully completed.

SQL> 
SQL> select empno, ename, job, sal
  2  from emp
  3  where job = 'CLERK';

     EMPNO ENAME      JOB              SAL                                                                                          
---------- ---------- --------- ----------                                                                                          
      7369 SMITH      CLERK            800                                                                                          
      7876 ADAMS      CLERK           1100                                                                                          
      7900 JAMES      CLERK            950                                                                                          
      7934 MILLER     CLERK           1300                                                                                          
      7369 SMITH      CLERK            800                                                                                          
...
      7934 MILLER     CLERK           1300                                                                                          

1024 rows selected.


Execution Plan
----------------------------------------------------------                                                                          
Plan hash value: 3956160932                                                                                                         
                                                                                                                                    
--------------------------------------------------------------------------                                                          
| Id  | Operation         | Name | Rows  | Bytes | Cost (%CPU)| Time     |                                                          
--------------------------------------------------------------------------                                                          
|   0 | SELECT STATEMENT  |      |  1024 | 23552 |     9   (0)| 00:00:01 |                                                          
|*  1 |  TABLE ACCESS FULL| EMP  |  1024 | 23552 |     9   (0)| 00:00:01 |                                                          
--------------------------------------------------------------------------                                                          
                                                                                                                                    
Predicate Information (identified by operation id):                                                                                 
---------------------------------------------------                                                                                 
                                                                                                                                    
   1 - filter("JOB"='CLERK')                                                                                                        


Statistics
----------------------------------------------------------                                                                          
          1  recursive calls                                                                                                        
          0  db block gets                                                                                                          
         98  consistent gets                                                                                                        
          0  physical reads                                                                                                         
          0  redo size                                                                                                              
      31902  bytes sent via SQL*Net to client                                                                                       
       1247  bytes received via SQL*Net from client                                                                                 
         70  SQL*Net roundtrips to/from client                                                                                      
          0  sorts (memory)                                                                                                         
          0  sorts (disk)                                                                                                           
       1024  rows processed                                                                                                         

SQL> 
SQL> select empno, ename, job, sal
  2  from emp
  3  where job = 'PRESIDENT';

     EMPNO ENAME      JOB              SAL                                                                                          
---------- ---------- --------- ----------                                                                                          
      7839 KING       PRESIDENT       5000                                                                                          
      7869 JACK       PRESIDENT       5000                                                                                          


Execution Plan
----------------------------------------------------------                                                                          
Plan hash value: 1472992808                                                                                                         
                                                                                                                                    
---------------------------------------------------------------------------------------                                             
| Id  | Operation                   | Name    | Rows  | Bytes | Cost (%CPU)| Time     |                                             
---------------------------------------------------------------------------------------                                             
|   0 | SELECT STATEMENT            |         |     2 |    46 |     2   (0)| 00:00:01 |                                             
|   1 |  TABLE ACCESS BY INDEX ROWID| EMP     |     2 |    46 |     2   (0)| 00:00:01 |                                             
|*  2 |   INDEX RANGE SCAN          | EMP_IDX |     2 |       |     1   (0)| 00:00:01 |                                             
---------------------------------------------------------------------------------------                                             
                                                                                                                                    
Predicate Information (identified by operation id):                                                                                 
---------------------------------------------------                                                                                 
                                                                                                                                    
   2 - access("JOB"='PRESIDENT')                                                                                                    


Statistics
----------------------------------------------------------                                                                          
          1  recursive calls                                                                                                        
          0  db block gets                                                                                                          
          5  consistent gets                                                                                                        
          0  physical reads                                                                                                         
          0  redo size                                                                                                              
        812  bytes sent via SQL*Net to client                                                                                       
        499  bytes received via SQL*Net from client                                                                                 
          2  SQL*Net roundtrips to/from client                                                                                      
          0  sorts (memory)                                                                                                         
          0  sorts (disk)                                                                                                           
          2  rows processed                                                                                                         

SQL> 

Now we get the ‘correct’ execution plans as the query returning 25% of the total rows uses a full table scan to reduce I/O. This is because the index access cost has not been artificially modified to favor such scans. We still get an index scan for the second query that returns 2 rows, which is as it should be.

What was good in earlier releases of Oracle may no longer provide benefit since changes in the CBO and statistics gathering could make such settings detrimental to perfrmance, as the above example illustrates. This example was also executed against an EMP table containing around 1.5 million rows with the same results; it isn’t difficult to realize that doubling the I/O arbitrarily isn’t a good idea. Sometimes the status quo shouldn’t be maintained.

It’s easy at times to pass through settings that once provided benefit in older releases of Oracle. Such settings should be examined and tested before passing them on to production unaltered as they may increase the work Oracle does to retrieve data. Testing is key; yes, that can prolong completing a migration to a newer release but it could prove invaluable in reducing I/O and improving performance, as illustrated here. And it may prevent the DBA from hunting down a performance problem that could have been avoided.

Sometimes it’s better to make a left-hand turn now and then.

November 14, 2014

In Deference

Filed under: General — dfitzjarrell @ 08:08

An interesting ‘problem’ surfaced a while ago, one where a user with zero quota on every tablespace could successfully create tables. Of course once it was time to insert data the inserts failed, but this was confusing the user creating the tables. The ‘problem’ stems from enabling deferred segment creation in the database. Let’s see how that can create a confusing situation,

Deferred segment creation allows tables and indexes to be created without physical segments which can be a double-edged sword as it allows objects to be created even if there is no space available for those objects in the specified tablespace. Just like the user who has no quota on the desired tablespace a false sense of security can result when the database reports ‘Table created’ or ‘Index created’. The data dictionary contains the object information so a query of DBA_TABLES, DBA_INDEXES or any of the %TAB% and %IND% tables and views will return the expected information. Querying DBA_SEGMENTS, however, may fail to return data since an object may not have any segments associated with it:


SQL> --
SQL> -- Prove the table exists
SQL> --
SQL> select table_name, tablespace_name
  2  from user_tables;

TABLE_NAME                     TABLESPACE_NAME
------------------------------ ------------------------------
KRONK                          USERS

SQL>
SQL> --
SQL> -- Now prove it doesn't exist
SQL> --
SQL> select segment_name, tablespace_name
  2  from user_segments;

no rows selected

SQL>

Looking at a session creating a table, owned by a user who has 0 quota on every tablespace, reveals this possibly confusing behavior:


SQL> 
SQL> --
SQL> -- See if deferred segment creation is enabled
SQL> --
SQL> show parameter defer

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
deferred_segment_creation            boolean     TRUE
SQL> 
SQL> --
SQL> -- We see that it is
SQL> --
SQL> -- Show quotas for user BLORPOFORST
SQL> --
SQL> @get_user_quotas.sql blorpoforst
SQL> with ttlbytes as (
  2  	select tablespace_name, sum(bytes) ttl_bytes, sum(blocks) ttl_blocks
  3  	from dba_data_files
  4  	group by tablespace_name
  5  ),
  6  userquotas as(
  7  	select
  8  	TABLESPACE_NAME,
  9  	USERNAME,
 10  	bytes,
 11  	nvl(max_bytes,0) max_bytes,
 12  	nvl(blocks,0) blocks,
 13  	nvl(max_blocks,0) MAX_BLOCKS
 14  	from dba_ts_quotas
 15  	where username = upper('&&1')
 16  )
 17  select
 18  s.tablespace_name,
 19  nvl(q.username, upper('&&1')) username,
 20  nvl(BYTES,0) bytes,
 21  case when nvl(q.MAX_BYTES,0) = -1 then s.ttl_bytes else nvl(q.max_bytes,0) end max_bytes,
 22  nvl(blocks, 0) blocks,
 23  case when nvl(q.MAX_BLOCKS,0) = -1 then s.ttl_blocks else nvl(q.max_blocks,0) end max_blocks,
 24  case when nvl(q.bytes,0) > 0 and abs(nvl(q.max_bytes,0) - nvl(q.bytes,0))  0 and abs(nvl(q.max_bytes,0) - nvl(q.bytes,0)) between 102400 and 10240000 then 'WARNING'
 26  	  else 'OK' end status
 27  from userquotas q full outer join ttlbytes s
 28  	   on (q.tablespace_name = s.tablespace_name)
 29  order by q.username, s.tablespace_name;
old  15:    where username = upper('&&1')
new  15:    where username = upper('blorpoforst')
old  19: nvl(q.username, upper('&&1')) username,
new  19: nvl(q.username, upper('blorpoforst')) username,

TABLESPACE_NAME                USERNAME                            BYTES  MAX_BYTES     BLOCKS MAX_BLOCKS STATUS
------------------------------ ------------------------------ ---------- ---------- ---------- ---------- -------
DFLT                           BLORPOFORST                             0          0          0          0 OK
INDX                           BLORPOFORST                             0          0          0          0 OK
SYSAUX                         BLORPOFORST                             0          0          0          0 OK
SYSTEM                         BLORPOFORST                             0          0          0          0 OK
UNDOTBS1                       BLORPOFORST                             0          0          0          0 OK
USERS                          BLORPOFORST                             0          0          0          0 OK

6 rows selected.

SQL> 
SQL> 
SQL> --
SQL> -- Absolutely no quotas on any tablespace
SQL> --
SQL> -- Try to create a table, which should fail
SQL> --
SQL> -- Succeeds due to deferred segment creation
SQL> --
SQL> create table kronk(ipso number, egrud varchar2(40))
  2  tablespace users;

Table created.

SQL> 
SQL> --
SQL> -- This will be confusing to users creating tables
SQL> --
SQL> -- 'Table' creates, but no data can be added
SQL> --
SQL> insert into kronk(ipso, egrud)
  2  values (1, 'Insert fails!!!!!');
insert into kronk(ipso, egrud)
            *
ERROR at line 1:
ORA-01950: no privileges on tablespace 'USERS'


SQL> 
SQL> --
SQL> -- Deferred segment creation allows the 'object' to
SQL> -- exist in the data dictionary even though the quota
SQL> -- prohibits any actual segments being created
SQL> --
SQL> -- Try this again, with an additional parameter
SQL> -- to the create table statement
SQL> --
SQL> drop table kronk purge;

Table dropped.

SQL> 
SQL> create table kronk(ipso number, egrud varchar2(40))
  2  segment creation immediate
  3  tablespace users;
create table kronk(ipso number, egrud varchar2(40))
*
ERROR at line 1:
ORA-01950: no privileges on tablespace 'USERS'


SQL> 
SQL> 
SQL> --
SQL> -- Now the table create fails, as expected
SQL> -- because segment creation was forced
SQL> --
SQL> -- Another way to do this is to disable
SQL> -- deferred segment creation at the session
SQL> -- level
SQL> --
SQL> -- This can also be done at the system level
SQL> -- without restarting the database
SQL> --
SQL> alter session set deferred_segment_creation=false;

Session altered.

SQL> 
SQL> --
SQL> -- Now the original create table statement will fail
SQL> --
SQL> create table kronk(ipso number, egrud varchar2(40))
  2  tablespace users;
create table kronk(ipso number, egrud varchar2(40))
*
ERROR at line 1:
ORA-01950: no privileges on tablespace 'USERS'


SQL> 

Besides confusing any users creating tables it can also take a DBA by surprise if, in earlier releases of Oracle, the DBA checked for object existence by querying the DBA_SEGMENTS view:


SQL> show parameter defer

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
deferred_segment_creation            boolean     TRUE
SQL>
SQL> create table yazzoo(plunk number, geepo varchar2(40), alpinga date)
  2  tablespace users;

Table created.

SQL>
SQL> select table_name, tablespace_name
  2  from user_tables;

TABLE_NAME                     TABLESPACE_NAME
------------------------------ ------------------------------
YAZZOO                         USERS
DUMMY                          USERS
SALGRADE                       USERS
BONUS                          USERS
DEPT                           USERS
EMP                            USERS

6 rows selected.

SQL>
SQL> select segment_name, tablespace_name
  2  from user_segments;

SEGMENT_NAME                                                                      TABLESPACE_NAME
--------------------------------------------------------------------------------- ------------------------------
DEPT                                                                              USERS
DUMMY                                                                             USERS
EMP                                                                               USERS
SALGRADE                                                                          USERS

SQL>

Notice that two tables, the one just created (YAZZOO) and one of the demonstration tables, BONUS, have no segments even though they do exist in USER_TABLES. I know of DBAs who use DBA_SEGMENTS to check objects and with deferred segment creation that may provide a list that doesn’t include every table a user has created. With deferred segment creation enabled older scripts that used DBA_SEGMENTS to successfully report on all objects need to be modified to use DBA_TABLES and DBA_INDEXES to return all tables and indexes regardless of whether segments are created or not; the older scripts can also be modified to show tables and indexes which have no segments as these example using USER_TABLES, USER_INDEXES and USER_SEGMENTS illustrate:


SQL> select table_name, tablespace_name, nvl(ttlbytes,0) ttlbytes, nvl(ttlext,0) ttlext
  2  from
  3  (select t.table_name, t.tablespace_name, sum(s.bytes) ttlbytes, sum(s.extents) ttlext
  4  from user_tables t left outer join user_segments s on (s.segment_name = t.table_name)
  5  group by t.table_name, t.tablespace_name);

TABLE_NAME                     TABLESPACE_NAME                  TTLBYTES     TTLEXT
------------------------------ ------------------------------ ---------- ----------
DUMMY                          USERS                               65536          1
EMP                            USERS                               65536          1
DEPT                           USERS                               65536          1
BONUS                          USERS                                   0          0
YAZZOO                         USERS                                   0          0
SALGRADE                       USERS                               65536          1

6 rows selected.

SQL>
SQL> select index_name, tablespace_name, nvl(ttlbytes,0) ttlbytes, nvl(ttlext,0) ttlext
  2  from
  3  (select t.index_name, t.tablespace_name, sum(s.bytes) ttlbytes, sum(s.extents) ttlext
  4  from user_indexes t left outer join user_segments s on (s.segment_name = t.index_name)
  5  group by t.index_name, t.tablespace_name);

INDEX_NAME                     TABLESPACE_NAME                  TTLBYTES     TTLEXT
------------------------------ ------------------------------ ---------- ----------
SALGRD_IDX                     INDX                                65536          1
YAZZOO_IDX                     INDX                                    0          0
DEPT_IDX                       INDX                                65536          1
EMP_IDX                        INDX                                65536          1
BONUS_IDX                      INDX                                    0          0

SQL>

Deferred segment creation can be very helpful when creating objects that may not be used immediately; it can also create problems when those objects are created by a user account with no quota on a tablespace or when the available space in a tablespace would prevent a segment from being created. The problem won’t become apparent until the first insert is met with failure because of a missing quota or insufficient available space. The DBA should get used to checking DBA_TABLES, DBA_INDEXES and DBA_SEGMENTS to verify the object can contain data. Also users who create tables should test an insert after the table is created to prevent any surprises later. Indexes would be tested with that same test insert; if the index is the ‘offending’ object the test insert will reveal this:


SQL> insert into kronk(ipso, egrud)
  2  values (1, 'Insert fails!!!!!');
insert into kronk(ipso, egrud)
            *
ERROR at line 1:
ORA-01950: no privileges on tablespace 'INDX'


SQL>

Deferred segment creation is, for the most part, a good thing as it allows tables and indexes to be created even when space isn’t immediately available. (Of course it’s not a good idea to create tables and indexes when it’s known there is insufficient space, but in the course of table/index maintenance space may be freed by dropping old versions of a table or by archiving and purging old data no longer needed for daily operation.) A good example for deferred segment creation would be migrating a table to a new definition; deferring segmnent creation allows the new table to grow as the data is inserted rather than creating a large extent where some of that allocated space may remain unused. Another benefit is the new table may be smaller initially than the original even though the number of rows has not decreased. Deferred segment creation can also be ignored, either by setting the deferred_segment_creation parameter to FALSE or by explicitly coding SEGMENT CREATION IMMEDIATE in the create table/create index statement, as shown in the examples above.

How to manage segment creation is a decision that should not be made without serious discussion between the development teams and the DBAs. It can generate a lot of create table activity in a database, creating various incarnations of a table before deciding on the final structure (a task usually left to the architects as an exercise on paper before any code is written and executed). It’s also possible that the development team isn’t aware of deferred segment creation; this can result in the ‘surprise’ reported at the beginning of this article when the first attempt to use the created table is executed. It may be better to see the error when the statement is executed rather than later, when data cannot be inserted because segments cannot be created.

Should you use deferred segment creation? That choice is one both the DBA team and development team need to make. Look at all aspects of the argument then decide accordingly. Remember that the ‘right’ answer is the one that’s suited to your environment. One size does not fit all. Choose wisely for the best fit for your environment.

Next Page »

The Rubric Theme. Blog at WordPress.com.

Follow

Get every new post delivered to your Inbox.

Join 893 other followers