Dr. Walley: This is safety nationally. 
When I talked about maltreatment reccurrence, where in '05 we were
looking at about 8.1% of maltreatment reccurrence, 
now we are down to about 6.7-6.8%.
I want to call this to your attention because a reduction repeat maltreatment 
is one of those four critical measures that the commissioner has noted.
It's also one we are making great progress on and think about this for a moment:
It is occurring simultaneously to a decrease in the number 
of children in out-of-home care.
As the number of children staying in safe, stable homes increases, 
maltreatment in care decreases.
A lot of times we've had this fallacy in our thinking that we
have to rescue children and that somehow they're safer being
swept into the systems of the state. 
But when we do that we see maltreatment in care rise.
So, it's almost one of those counter-intuitive outcomes.
We're going to look in a moment and see what's going on in
Oklahoma with that maltreatment, repeat maltreatment.
If you would, let me call your attention now to a few of the handouts that you have.
I think we've already looked at the rate of children entering out-of-home care.
There is a handout that is entitled "Jurisdiction Data Report"
and it is about a three-page hand out.
And it's one of the most informative ones, I think, 
that I have provided for you today, because I want to tell you what it says.
And again, you remember my opening remarks that 
everyone is entitled to their own opinion, no one is entitled to their own facts.
These are the data facts and I want to present them to you 
as a preliminary comment to Dr. Steib's report then.
You'll notice in Oklahoma... and I'm going to look at the year 2007
and that second graph you have in the middle of that first page.
In 2007, you had roughly 11,800 children in out-of-home care
with an out-of-home care rate of 13.7 per thousand. 
That's an extraordinarily high number. Over double the national average in '07.
Oklahoma was taking tremendous numbers of kids into its custody care.
Over the course of the last three years, moving up, let's say through FY '10, 
you now have approximately 8,000 children in out-of-home care, 
a reduction of 31% and your rate in out-of-home care has been
reduced from 13.7 to 8.4 per thousand.
That is something to celebrate in our minds, in Casey Family Program's minds, 
because again, you're going to see in a moment the relationship of that 
to the repeat maltreatment issue.
Eight point four, now that's still high and that is an area that we have
been quite up front with the Director that we would love the
opportunity to work with the Commission and the Department to continue strategizing,
"What can we do to deal with still the high rate of children in out-of-home care in Oklahoma?" 
Because as your rate has come down, so has the national average, too.
It's 5.2 per thousand.
So, instead of being over twice, you are not twice as high any longer:
8.4 per thousand compared to 5.2 per thousand.
There was another map I had for you, color-coded map 
in your packet called "Rate of children under 18 in out-of-home care."
And you will see that in this one, Oklahoma is among one of the darker states, 
that has the higher rate in care compared to peer states around the country
compared to the national average.
Let me mention about... I don't think I have this one, 
but your rate of children entering out-of-home care, it has improved.
The national average is 3.3 per thousand children in Oklahoma who enter out-of-home care.
Excuse me-nationally, 3.3 per thousand. It's 4.6 per thousand in Oklahoma.
So, children are still entering in at a higher rate
into out-of-home care here in Oklahoma than they are nationally.
Commissioner: Is there a chart on that?
Dr. Walley: I don't think that I have the chart on that one.
I apologize, we can get that for you.
Well-I do have that-rate of children entering out-of-home
care, it was uh...well, let's see...yeah.
Rate of children entering out-of-home care was this one right here, there was that one.
It was just the one of the two maps.
Let me ask you to turn, look at the bottom, if you don't mind, too.
This is a reflection of that relationship between entries and exits 
and you will see one of the important figures at the bottom 
is that 90% of all of the children that are exiting are exiting to permanency.
The national average is only 85%.
So, when a child exits from care in Oklahoma, 90% of the time they're exiting to permanency.
Nationally that's 85%.
If you'll turn to the next page in the upper left-hand corner
there is a graph called "Absence of Maltreatment Reccurrence."
This is a critical data point. One of the most important.
You see that Oklahoma between '05 and '08 was really
significantly below the national average in absence of
maltreatment reccurrence.
But there were some practice changes in Oklahoma. 
At the same time, there have been reduction in number of children
in out-of-home care and the absence of repeat maltreatment
began to trend the right way 'til now. 
You'll notice the absence of repeat maltreatment
is at the national standard - the aspirational standard
of 94.2% of children are absent of maltreatment recurrence.
To the right on the graph of that top page you'll notice
something about entries by first entry and re-entry.
That gets at the issue of when children are entering Oklahoma's
system, how many of them are coming back? They had exited and returning?
This graph shows that 15.6% of the entries were re-entries, 
which is below the national average of 20% being re-entries.
At the bottom of that page are just some of the data points
that we internally at Casey are tracking based on some
particular work efforts we have here.
The percentage of children entering foster care are
decreasing by 10% a year; it was 8.6%, so on the bottom left 
we have a graph that about... the issue of Oklahoma county 
and trying to shorten by one month the mean time for youth reaching permanency.
That's very challenging, and Howard referenced it earlier in that, 
as we get more children home, you have those that are some 
of the more challenging cases remaining in care, 
so that's not a very responsive measure we don't think 
and we are going to look at modifying that.
One of the other ones that we don't have on here, 
but I want to call your attention to, 
is what we consider measure of placement stability 
and that is the percentage of children re-entering care 
within twelve months of reunification.
You send a child home with a family, what is the probability
that they're going to be back in your system within twelve months?
Oklahoma, the data shows 8.3% in Oklahoma compared to 
the national average of 12.3%.
On the right-hand side, just quickly, one of the issues that
I know we had been engaged with in Oklahoma on relates to 
the use of congregate care, shelter care.
And because of some work that we've been doing... 
This is on... do you see that top graph where it says, 
"36% reduction in the number of children under six placed 
in shelters over two years"?
I don't know if you realize it, but that is, again, something to be commended.
The use of shelter care, research shows, 
particularly for children six and under is not best practice.
So that's one of the areas that we have been fortunate enough to
be involved with in working with the state of Oklahoma.
There're many other work efforts.
But what I wanted to do is to tell you that back earlier this year, 
we had a request from the Oklahoma Department of Human Services 
to assist them with an evaluation among the other things that we are doing. 
And we're... we feel like we're doing quite a bit here 
just before I get to that evaluation.
We have invested about $800,000 in resources into Oklahoma in 2011 
and there will be a continued investment of those kind of resources 
and that's not checks being written.
We do support opportunities for training. 
We do come in and we've trained 850 workers in Oklahoma in permanency values training 
and permanency orientation around a methodology called "Permanency Round Tables."
Where we're identifying those most challenging cases out there that
have been stuck in out-of-home care for years 
and what we can do to safely find permanency for those children.
That's going on statewide. Dr. Steib has been leading that effort for us here.
By the way, I wanted to give you a bit of context too...
When I told you about the 31% reduction in Oklahoma's number of children
in out-of-home care in the last few years.
While that is note worthy, Florida has had a 40% reduction,
Georgia - 48%, Maryland - 45%, New Jersey - 40%, Ohio - 31%, 
Virginia - 29%, California - 25%.
So, you begin to see that this is not an issue particular to Oklahoma.
This is a national trend.
Getting back to that first slide where people are recognizing
that we have had a system that does not prioritize families
and children in families safely.
And we have reaped the harvest of that, but that's changing.
We were asked early this year, as I was noting a little bit earlier, 
to review the safety assessment guidance and practice in Oklahoma DHS, 
related to House Bill 17 that passed in 2009 changing the standard for
removal and what were the effects of that on practice?
What was the practice model's effect on practice and implementation?
What could we learn and what could we do to help Oklahoma
with this ongoing self-assessment which is a sign 
of any system that hopes to be healthy and productive, 
is its ability to self-assess. Its ability to use data.
Its ability to take feedback from commissions and other oversight bodies 
and then to use that to change practice, to change personnel, to change
whatever needs to be changed to continue on the quality improvement.
That's what we do, that's what any system does, 
that's what any organization does, that's what any team does, 
and so we're proud to be a part of doing that.
I lived in Auburn, Alabama and we lost our first game in 17 games a week ago.
Commissioner: Waa...
(Laughter) 
Dr. Walley: The reason I tell you that is because there was an immediate
concern that, "what's happened? This is not supposed to happen. 
We need...what's going on?"
And so there had to be an assessment by the coaching staff 
of everything ongoing in that organization and make adjustments.
And see was it policy? Was it system? Was it training? 
Was it personnel? Was it...? Was it...? Was it...?
Because the outcomes aren't what they wanted.
Now the reason I share that metaphor with you is this: 
Any system that wants to be premiere has to engage in that.
Any system that suffers trauma, as all systems will, 
has to be prepared to take a look and see.
What are we doing? How do we need to respond? And never overreact.
To look at the trauma and look at the successes with equal concern 
and not judge the system by either at any one time.
But to be reflective and to make those decisions on how to improve the system.
We appreciate Oklahoma's DHS' willingness to allow us to be part of that.
And the evaluation you will now hear from Dr. Steib and then
both of us will be available to answer questions subsequently.
Thank you very much.
Commissioner: Thank you Dr. Walley.
(Applause) 
Commissioner: Now obviously, Dr. Steib, you haven't had any traumas 
since you are from LSU?
Dr. Steib: That's right.
We'll see what happens on Saturday, now won't we?
Thank you Page, and Mr. Chairman, and the ladies and gentlemen of the commission.
Thank you for the opportunity to be here.
I am just going to briefly review for you 
and to give a little bit of discussion and interpretation concerning 
the report that we did of the Safety Assessment Review that my colleague, 
Barry Salovitz, and I conducted at the beginning of this year.
And it's my understanding that you have all been given copies of that report.
Is that correct? Okay. Alright.
So as Page said, this was done at the request of the DHS leadership 
as a result of the legislation that was passed in 2009 which had, as we all know,
changed the criterion for removal of children from their families.
DHS had then embarked upon some changes in policies and
processes and procedures and assessments tools and so on.
And those had been implemented and they thought
that they were at a point where they needed someone to take a look,
do some inquiry, and see how those changes were being affected 
and how they were impacting practice and the safety of children in Oklahoma.
So that is what my colleague, Barry Salovitz, and I did.
As Page mentioned earlier, Barry Salovitz is nationally known 
in the area of child safety assessment.
He's the former Director of the National Resource Center on Child Maltreatment, 
he is the developer of one of the very first and best recognized 
safety assessment tools in the nation and one that is a forerunner 
of the many safety assessment tools that are used around the country today.
So, he was very helpful and a very significant factor 
in our being able to conduct this review.
The review was conducted during the first quarter of 2011.
We first reviewed a set of documents that the administration provided for us.
Documents that they felt best reflected the changes that had been made in policy 
and practices and processes here in Oklahoma to comply with the 2009 legislation, 
and those documents are detailed in your report.
Following that document review we did interviews with a group
of fifteen staff, we asked to interview staff at all levels.
We wanted staff who were fairly experienced, 
we wanted to interview staff in at least one urban area 
and one rural area and we did that with a series of guided, 
but really open and flexible interviews that lasted anywhere
from about 20 minutes to about 45 minutes a piece.
So that was our methodology.
Our questions in our interviews centered on what they thought was the impact 
of the imminent safety threat criterion,
whether they thought it compromises childhood safety in Oklahoma,
and how they viewed the strengths and also the concerns
with regard to the new safety assessments and safety planning processes.
So, what did we find?
And I'm going to just briefly go through and talk about the documents 
and I'll talk a little bit more about the concerns, so that you'll
understand what the basis of those was.
But first, and I want to say that overall with regard to all of the documents: 
the practice model guide, the policy guidances that we were provided, 
the training materials that we were provided...
overall we found all of these documents to be generally in accord with what we
consider to be sound practice in child protective services.
We started with the practice model guide, which is a comprehensive document.
We were focused, because that was our charge, primarily on the safety assessment portion 
of that, but that is a comprehensive document that speaks to, really, 
all aspects of child protective services and child welfare practice in Oklahoma.
It was developed with input and participation from DHS child welfare staff here, 
but with guidance and consultation from some 
of the Federal National Resource Centers and their expertise.
Again, we found many strengths in the practice model guide, 
we only had a few concerns.
One was that, while the guide does speak to issues of child vulnerability, 
we felt that it could focus more on discussion of vulnerability 
throughout the development of a child.
The focus was primarily on vulnerability of young children, 
and certainly we understand that. Young children are uniquely vulnerable. 
Most of those serious injuries that we see in child maltreatment 
occur with younger children... so that focus is understandable.
But we also know that older children can be vulnerable 
to different kinds of maltreatment and different kinds of exploitation.
So, we would have liked to have seen some more discussion of that.
We also thought that there could be more guidance with regard to situations
where there are multiple risk factors in a case or in a report, 
but the situation at that time does not meet the threshold 
for present danger or imminent danger.
Commissioner: If you don't mind me asking a question right here?
A report that was done recently by, I think the legislature, that authorizes by Zeller...
Dr. Steib: Hornby Zeller.
Commissioner: They made distinctions between safety issues and risk factors.
Safety issues being immediate things that need control...
Dr. Walley: That's correct.
Commissioner (continues): ...and risk monitoring evaluations and so on...
are you making the same kind of distinctions?
Dr. Walley: Yes. Yes, we are. 
And generally speaking, and I believe this is reflected in the Hornby Zeller report as well, 
we do when we are thinking about intervening in a family, 
particularly that child removal, we do want to be focused on issues of safety.
The practice model guide encourages those situations in which a child is found 
to be unsafe to be retained under the purview of CPS and for there to be 
some sort of intervention that, preferably that, the safety threats be mitigated
without the child having to come into custody.
But if that cannot occur, then for the child to be removed 
and placed within protective custody or for a safety plan to be initiated 
that allows the child, at least for a time, to be in an alternative setting 
such as in the home of an extended family member.
And it encourages that all of the situations in which 
longer term risk factors, such as those you referenced
from the Hornby Zeller Report, or with that distinction between
risk and safety, to be referred to community resources.
Generally speaking, that's in keeping with good practice.
We know though that in Oklahoma, as in every state where I've worked, 
there are in some communities limitations on resources, their scope, 
their availability, their accessibility... 
sometimes families are less willing to avail themselves of those resources.
So, it does not always in every case fully protect children
if they're only pursued within the child welfare system,
if the child is at this moment unsafe. For the most part, yes. 
But in some cases, probably few cases,
and I will refer to that later on as I go through the presentation,
that may not be sufficient.
And so, we would have liked to have seen some more discussion of that 
and of the application of risk in the practice model guide.
With regard to the policy memoranda and the associated
documents that we reviewed, again we found them to be comprehensive.
We thought they laid out quite well the scope of an
investigation as it should be conducted.
We really... our concerns regarding those policy documents were very minimal. 
We had really only one that's mentioned in the report.
And that was that there was one document that indicated that
interviews with parents, it used permissive language that may
include interviews with parents or other persons responsible 
for the care of the child. 
And we questioned that, but that was our only question 
or concern regarding the policy documents that we were given to review.
With regard to the assessment of child safety tool, 
and I will say this is possibly the most comprehensive assessment 
of child safety instrument I have reviewed,
it is quite lengthy and it is quite thorough.
It contains definitions of various levels of danger. It talks about vulnerability. 
It has a whole series of guidance questions that investigators can use 
to elicit information about safety threats.
We had a few concerns, again as in the practice model guide, 
we had that same concern about the discussion of vulnerability could have been more thorough.
We thought that perhaps there was a need for greater clarity
between what's considered present danger and impending danger.
Although we understood, following the issuance of the report, 
that that is being addressed in training here in DHS now.
And again, we also had the concerns about the fact that the protective capacities, 
as they're addressed in the safety assessment document used here in Oklahoma, 
are only considered after the child is determined to be unsafe.
Our recommendation was that protective capacities, factors that might 
mitigate safety threats, be considered as part of the "safe or unsafe" decision.
In Oklahoma, protective capacities are not considered in making the unsafe decision, 
they are only considered in the response.
Now I'll tell you that, although we question that, 
the Oklahoma practice is a more conservative practice.
So, their guidance calls for going ahead and making a decision about
whether a child is safe without considering any factors that might mitigate that.
It is a more conservative response and one that...
this is one of those areas where reasonable people can have different views.
And one that, in the judgment of DHS, was appropriate at that point in the
implementation of this safety assessment process and we were understanding of that.
Alright, and then with regard to training and related documents,
again... generally in accordance with sound practice.
They... we were provided with a well-documented training
on what's called critical thinking, that targets supervisors, 
which I show here is a strength, because what we know is that 
in child protection and child welfare practice, 
good, solid supervision is absolutely the lynch pin.
You cannot have effective practice around child safety and child wellbeing 
unless case workers have very strong and solid supervision. 
And so we viewed it as a strength that this training was being provided 
to supervisors in Oklahoma.
There was one document that we reviewed that we thought might be a little confusing 
to staff because, again, it recommended the process of including protective factors 
in making the safety decision and that's not the policy here. 
That was a minor finding and then just that some of the documents
that we were provided with... things like Power Points and notes, 
so it was difficult for us to really fully assess what actually might occur when
they were delivered in training. 
Alright. Now we move to the interviews.
We talked to all of the staff that we interviewed about the imminent danger standard 
and how that affects child protection practice in Oklahoma.
Consistently they told us that they did not feel that the adoption 
of the new standard overall had compromised child safety, in most instances.
They felt that in fact most of the changes in practice that had ensued 
in the implementation of the legislation in the new standard had been positive ones
as they compare them to former practice.
They talked at length in almost every instance about the fact that currently 
they are trained and encouraged to look more deeply at presenting situations,
that prior to the enactment of current policies and procedures 
they used what was called an incidence-based response in Oklahoma.
So that really the focus was just on the referral 
and the presenting information and they were not encouraged 
to look for underlying factors that might in fact suggest that a danger 
to a child could escalate beyond just what is observed in an investigation, 
if one does not look more deeply.
So overall staff perspective on the new standard was positive.
There were no concerns voiced specifically about that.
Now as we went on and talked more in detail about specific ways 
in which child safety might be impacted as the standard is applied.
Again, people felt that the more narrow criteria was needed 
that there were children who were removed unnecessarily under the old standard, 
but that there were those few cases, and I referenced this
when I talked about the practice model guide a few minutes ago, 
where there are multiple risk factors and several staff felt that they needed more clarity, 
better guidance around how to handle those situations where no child was unsafe, 
there was no current, present dnager, no impending danger that could be identified, 
but there were multiple risk factors in a situation, such that it could change 
and risk or danger could escalate fairly quickly if nothing was done.
Insofar as the process of assessing child safety,
overwhelmingly, staff felt that the process was a good one.
They talked to us about some bumps in the road about implementation, 
because things had been done pretty quickly, but that now training had been completed.
They were becoming more proficient in using the tools
and the processes and they truly felt that they were doing more thorough investigations.
They did tell us that this
 
process takes more time, and
 
that it continued to take more

time even though they were for 
the most part over the learning 
curve, if you will. 

There were also some issues with 
regard to documentation and 
duplication of documentation 

that needed to be addressed, and 
it's our understanding that 
those 
have been dealt with since 
the time that we issued this 
report. 

Also, some expression of 
concerns about inconsistency of 
skills across staff.

We know that you have a fair 
amount of turnover in some of 
your offices here in Oklahoma, 
so 
that... those comments weren't 
unexpected. 

With regards to using safety 
plans. 

Again, think in terms of a 
safety plan is applied only when 
a child has already been found 
to 
be unsafe in some way. 

And the safety plan then is 
crafted and enacted to mitigate 
the danger to the child, 
yet allow the 
child to remain at home or with 
the extended family. 

Overall, staff felt that was 
a good thing, that children were 
better off 
if removal can be 
avoided, just as we heard Page 
talk about a moment ago, 
but 
that creating and monitoring 
safety plans is challenging. 

It requires time, it requires 
skill, it requires 
resources... 
and that not all 
staff are equally... as well 
able to do that and to do that 
effectively. 

Continuing then with comments 
about staff skills and knowledge 
and capacity, 
specifically as 
it related to safety 
assessment and planning,
almost everyone we talked with 
told us that training has 
improved 
with the enactment of 
the new processes, and that they 
have had mentoring and coaching, 

which has been very helpful, so 
that there is ongoing support in 
helping case workers apply 
these new 
practices and processes. 

And again, they express concerns 
about turnover so that at any 
given time 
there are likely to 
be new and relatively 
inexperienced case workers in 
offices. 

That, also because of high 
turnover, that people may move 
into supervision 
without the 
optimal level of experience. 

That in some offices the ratio 
of supervisors to case workers 
was fairly high. 

From one supervisor to five case 
workers all the way up to one to 
nine. 

One to seven being the most 
common one that was cited to us. 

That there was a greater need 
for more skill building in 
training that older employees 

also needed to be brought into 
training. 

And that when reviews are done of 
situations where there have been 
serious injuries 
of children or 
deaths of children in child 
protection cases that there 
needs to be 
a systematic way of 
using and incorporating the 
learning from those analyses 

into policy and training. 
Again, this was not, this was a 
very focused review, so this was not in 
any way exhaustive 
in terms of 
looking at the full spectrum of 
child welfare services here. 
Much of the policy was still 
undergoing revision at the time 
we did this analysis and report, 
so we did not do an in-depth 
policy analysis. 

We reviewed the documents that 
were provided to us and we 
didn't do case reviews 
or 
interviews with service 
recipients and stake holders and 
so on. 

So, our coverage of areas was 
fairly limited due to our time 
constraints, 
so those are what 
we consider to be the 
limitations of this report. 

We did make several 
recommendations as you can see 
we made some, 
and these 
are fairly minor in my view, 
recommendations concerning 
revisions to this...
is the AOC, is 
the Assessment of Child Safety. 

We just made some suggestions 
about some relabeling and for 
the most part 
more minor sort of 
technical adjustments. 

Being sure that terms, specific 
terms particularly those related to 
safety response, 
were clear to 
staff. 

We again, followed up on some of 
the interview comments about 
duplication in documentation 
and 
the fact that these new 
processes were requiring more 
time. 

We had some recommendations 
pertaining to that, with regard to 
the documentation issue. 

Also, trying in the future... and I 
understand that there is some 
plans being 
made to do this,
that there 
would be a workload analysis to 
really get at with these 
changes, 
how has that impacted 
the amount of time that is 
required now to do a child 
protection investigation 
as 
compared to the time it required under the old 
policy and procedures? 
Again, recommended that priority 
be given to the coaching and 
mentoring, 
which we know is so 
important in allowing the staff 
to gain proficiency in this type 
of assessment. 

And we made some recommendations 
that you can see here with 
regard to specific factors 
that 
needed to be focused on in that 
coaching and mentoring. 

That there needed to be people 
available in each area 
who... 
with expertise to provide 
consultation and practice model 
and its implementation. 

That this issue of the fact that 
there are some families with 
multiple risk factors, 
where the 
child is not yet determined to 
be unsafe, that that needed to 
be addressed 
with regard to 
guidelines and training and 
insuring that staff are clear 
on 
how to respond in those 
situations. 

And that there should be some 
associated revisions to the 
practice model guide to reflect 
that. 

And also, to look at the need in 
some situations possibly for 
court ordered services 
even when 
children remain in the home if 
the family is not participating 
voluntarily. 

Commissioner: Question? 

Dr. Steib: Yes? 

Commissioner: Sue, the 
recommendations on page twelve 
and thirteen are many, 
but could 
you provide just two that are 
primary recommendations as you 
finish your report? 

Dr. Steib: The big finding really in 
this... well, I might say two. 
I will start though with  what I 
was going to say. 

To us, the big finding here was this 
lack of clarity in these 
admittedly fairly rare 
situations, 
but in these cases 
of high risk. 

So that in my mind and I think 
in Barry's also, in our 
discussion, 
the discussions that 
we had after we completed this 
review with the administration 
and DHS, 

that was the first thing that we 
felt needed to be addressed. 

There were clearly some staff 
who were concerned about the 
fact that, while it may not be 
many children, 
it could 
still... that those situations 
could still be fairly 
significant 
and that they needed 
more clarity. 
So, that was the one. 

The other was that doing work 
this way is more time consuming 
and it's more difficult. 

I mean this is hard work folks. 

I mean these situations are 
complex and that staff... that's 
why we made the recommendation 

for the coaching and the 
mentoring. 

That there needs to be a 
sufficient number of people at 
the front lines of practice 
who 
truly have strong assessment 
skills, who understand the 
distinctions between risk and 
safety, 

and who can apply good critical 
thinking and who can support 
staff consistently in doing 
that. 
But it's clearly taking 
more time. 

That's the reason for our 
recommendation for the workload 
analysis 
and for the continuing 
emphasis on having people at the 
front lines of the practice 
who have the expertise, who can mentor and support those 
who are just coming on
who maybe 
still are lacking in experience. 

Commissioner: Thanks. 
Dr. Steib: 
Yes?
Commissioner: OK. 
So as I understand it, with your 
organizations help 
and 
I've been telling your partner 
here that for fifteen years and 
I want to tell you, 
well done 
and that I appreciate you all 
jumping in and helping here.  

Okay, the request here that your 
document says here, stemming 
from child deaths occurring in 
the late 2010, 
and which 
resulted in (inaudible) that raised 
questions concerning the 
policies 
that had already been 
established between 2009 and 2010 
from that legislation,correct? 

Dr. Steib: That's correct. 

Commissioner: But as I 
understand it you all have not, 

or did I see that correctly when 
I saw somewhere in there 
where you all had not 
actually 
looked at some actual cases or 
anything? 

Dr. Steib: We did not do case 
file reviews in this study. 
No we did not. 

Commissioner: As I understand it 
you are continuing to jump in 
and help us along? 

Is that correct? 

Dr. Steib: That's correct. 

Commissioner: So would you all 
be available then to actually 
examine? 

Since you all were called in 
because of the child death 
issues, would you be available to 
actually 
take a look at some of 
the child death cases in light 
of the policies that are going 
on 
and help us kind of determine 
whether or not our policies, 
rules, and regulations are 
working? 

Dr. Steib: We've not had that 
discussion, we have not been 
asked to do that. 

That's potentially 
something that we could do. 

Dr. Walley: Check case 
reviews... 

Dr. Steib: Yes, we have, Page was 
just mentioning, we have been 
very much involved 
in doing 
reviews of cases of another sort 
in Oklahoma this year. 


And those are situations of children 
who have been in care for a 
longer period of time 
and who 
are in need of permanency. So we have done some case 
reviews. 
I personally have sat 
at the table with many of your 
staff out in the counties around 
the state 
and discussed some 
very difficult and complex 
cases. 

We have not done a review of 
child protection investigations 
per se. 

Commissioner: Sue. 
Follow up to Wes' comments on 
page eleven, at the top. 

You say currently according to 
earlier reports, that these instances are 
investigated, 
but there is not a 
clearly defined and implemented 
process reusing the learning 

from the investigation to inform 
changes in training and or 
policy. 

Dr. Steib: Right. 

Commissioner: Do you find that 
to be pervasive from your 
interviews? 

Dr. Steib: No. 
No. 
That was mentioned by probably 
some of the management 
staff that we interviewed. 

It was not pervasive. 

Chairman: Any other 
questions?
Commissioner: What about 
adherence to the policies and 
the quality of training 
so it 
sticks with front line workers? How do you go about...or working 
with that? 

Dr. Steib: That, that, was 
really looking at adherence to 
policy was beyond 
the scope of 
this review except as was 
indicated by the number of 
comments that we heard 
in the 
interviews with regard to people 
saying things like, "
Given the turnover, 
given the 
number of new staff, given the 
difficulty of doing this work" 

that there were concerns that 
there were inconsistencies. 

And those comments were not 
surprising to us. 

We would have expected that at 
that point in the implementation of new 
processes like these, 
that, that 
would probably be the case. 

Commissioner: Can you assess or 
did you look at the level 1 
turnover and... 

Dr. Steib: I have not looked at the actual data 
currently... what it is currently, 

but what we were told at that 
point was that there were 
certain places in the state 

where turnover was a particular 
concern at that time. 

Commissioner: More of the front 
line than the supervisors? 

Dr. Steib: Yes, and at that time 
the Oklahoma City area was one 
in which there was particular 
concern. 

Director Hendrick: That was... at that 
time, that was when we had 
switched out part of the changes 

in the law that we were 
realignment in Oklahoma County 
and Tulsa County 
and so part of 
that was people getting their 
jobs changed was part of that 
realignment, 
so you had some 
changes, which you would expect. 
It has pretty much stabilized 
since that time... 

Dr. Steib: Good. 
Other questions? 

Dr. Walley: I was just going to 
follow up with one of the 
comments that Sue was making about, 
Commissioner Dow I think 
your question, about policies 
versus implementation of 
policies 
and that is one of the 
areas that we are... 
this was not 
designed to specifically get at 
that depth. 

I think one of the things that 
we did determine is that the 
policies and the plan are sound. 

The execution of the plan is 
something that has to be 
continually evaluated. 

Having led a system that was one 
of only two that's ever come 
out of federal oversight...
the 
reforms that are under way here 
and that we are pleased to be 
investing in 
and working with 
along side you on are 
multi-year. 

And that sometimes is difficult 
for those of us that want rapid 
change, 
to fully embrace, but 
the systematic change like we were 
showing on the data 
and the 
responses of the data to the 
practice changes, 

and hopefully the implementation 
of the practice that appears 
sound, 
is something that we 
are... we don't invest imprudently. 

And I'm responsible to my 
Executive Vice President and to 
our Board of Trustees to being able 
to say, 
"These are the 
jurisdictions where we believe 
with our investment of our 
resources 
and personnel we can 
assist them in achieving the 
vision for safe permanent homes 
for children 
and reducing the 
shame of having so many children 
raised out-of-home and aging 
out." 

This is a place where we have 
decided we are going to invest, 
they've decided that they want 

to continue to invest, because 
we see the plan is one that's 
sound. 

I read this recently and I just 
wanted to share one comment and 
then I'll step aside. 

You might have other questions. 

"The most successful child 
welfare agencies have strong and 
sustained leadership 
that has a 
clear and unwavering vision. 

They also have the commitment of 
political leaders, community 
partners, 
and the courts to 
undertake the multi-year 
endeavor required to improve 
outcomes." 

That is something that we adhere 
to and believe in.