//
// Licensed to the Apache Software Foundation (ASF) under one
// or more contributor license agreements.  See the NOTICE file
// distributed with this work for additional information
// regarding copyright ownership.  The ASF licenses this file
// to you under the Apache License, Version 2.0 (the
// "License"); you may not use this file except in compliance
// with the License.  You may obtain a copy of the License at
//
//   http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing,
// software distributed under the License is distributed on an
// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
// KIND, either express or implied.  See the License for the
// specific language governing permissions and limitations
// under the License.
//
[[provisioning-pull]]
==== Pull

Pull is the mechanism used to acquire identity data from Identity Stores; for each external resource, one or more
<<tasks-pull,pull tasks>> can be defined, run and scheduled for period execution.

Pull task execution involves querying the external resource for all <<mapping,mapped>> <<anytype, any types>>, sorted
according to the order defined by a custom implementation of
ifeval::["{snapshotOrRelease}" == "release"]
https://github.com/apache/syncope/blob/syncope-{docVersion}/core/provisioning-api/src/main/java/org/apache/syncope/core/provisioning/api/ProvisionSorter.java[ProvisionSorter^]
endif::[]
ifeval::["{snapshotOrRelease}" == "snapshot"]
https://github.com/apache/syncope/blob/3_0_X/core/provisioning-api/src/main/java/org/apache/syncope/core/provisioning/api/ProvisionSorter.java[ProvisionSorter^]
endif::[]
or its default implementation
ifeval::["{snapshotOrRelease}" == "release"]
https://github.com/apache/syncope/blob/syncope-{docVersion}/core/provisioning-java/src/main/java/org/apache/syncope/core/provisioning/java/pushpull/DefaultProvisionSorter.java[DefaultProvisionSorter^]
endif::[]
ifeval::["{snapshotOrRelease}" == "snapshot"]
https://github.com/apache/syncope/blob/3_0_X/core/provisioning-java/src/main/java/org/apache/syncope/core/provisioning/java/pushpull/DefaultProvisionSorter.java[DefaultProvisionSorter^]
endif::[]
.

Each entity is then processed in an isolated transaction; a retrieved entity can be:

. _matching_ if a corresponding internal entity was found, according to the <<mapping,mapping>> of - or the
<<policies-pull,pull policy>> set for, if present - the enclosing external resource;
. _unmatching_ otherwise.

Once this has been assessed, entities are processed according to the matching / unmatching rules specified for the pull task:
by default, unmatching entities get created internally, and matching entities are updated.

.Matching Rules
****
* `IGNORE`: do not perform any action;
* `UPDATE`: update matching entity;
* `DEPROVISION`: delete external entity;
* `UNLINK`: remove association with external resource, without performing any (de-)provisioning operation;
* `LINK`: associate with external resource, without performing any (de-)provisioning operation;
* `UNASSIGN`: unlink and delete.
****

.Unmatching Rules
****
* `IGNORE`: do not perform any action;
* `UNLINK`: do not perform any action;
* `ASSIGN`: create internally, assign the external resource;
* `PROVISION`: create internally, do not assign the external resource.
****

[[pull-mode]]
[TIP]
.Pull Mode
====
The Identity Store can be queried in different ways, depending on the _pull mode_ that is specified:

****
FULL RECONCILIATION:: The complete list of entities available is processed.
FILTERED RECONCILIATION:: The subset matching the filter (provided by the selected implementation of
ifeval::["{snapshotOrRelease}" == "release"]
https://github.com/apache/syncope/blob/syncope-{docVersion}/core/provisioning-api/src/main/java/org/apache/syncope/core/provisioning/api/pushpull/ReconFilterBuilder.java[ReconFilterBuilder^])
endif::[]
ifeval::["{snapshotOrRelease}" == "snapshot"]
https://github.com/apache/syncope/blob/3_0_X/core/provisioning-api/src/main/java/org/apache/syncope/core/provisioning/api/pushpull/ReconFilterBuilder.java[ReconFilterBuilder^])
endif::[]
of all available entities is processed.
INCREMENTAL:: Only the actual modifications performed since the last pull task execution are considered. This mode
requires the underlying connector bundle to implement the ConnId `SYNC` operation - only some of the available bundles
match this condition. +
*This is the only mode which allows pulling delete events*, which may end up causing the removal of internal entities.
****
====

[[pull-templates]]
[TIP]
.Pull Templates
====
With every <<tasks-pull,pull task>> it is possible to add a template for each defined <<anytype,any type>>.

As the values specified in the template are applied to pulled entities, this can be used as mechanism for setting
default values for attributes or external resources on entities.

A typical use case is, when pulling Users from the external resource `R`, to automatically assign `R` so that every
further modification in Apache Syncope to such Users will be <<propagation,propagated>> back to `R`.
====

===== PullActions

The pull process can be decorated with custom logic to be invoked around task execution, by associating
pull tasks to one or more <<implementations,implementations>> of the
ifeval::["{snapshotOrRelease}" == "release"]
https://github.com/apache/syncope/blob/syncope-{docVersion}/core/provisioning-api/src/main/java/org/apache/syncope/core/provisioning/api/pushpull/PullActions.java[PullActions^]
endif::[]
ifeval::["{snapshotOrRelease}" == "snapshot"]
https://github.com/apache/syncope/blob/3_0_X/core/provisioning-api/src/main/java/org/apache/syncope/core/provisioning/api/pushpull/PullActions.java[PullActions^]
endif::[]
interface.

Some examples are included by default, see the table below.

[cols="1,2"]
|===

|
ifeval::["{snapshotOrRelease}" == "release"]
https://github.com/apache/syncope/blob/syncope-{docVersion}/core/provisioning-java/src/main/java/org/apache/syncope/core/provisioning/java/pushpull/ADMembershipPullActions.java[ADMembershipPullActions^]
endif::[]
ifeval::["{snapshotOrRelease}" == "snapshot"]
https://github.com/apache/syncope/blob/3_0_X/core/provisioning-java/src/main/java/org/apache/syncope/core/provisioning/java/pushpull/ADMembershipPullActions.java[ADMembershipPullActions^]
endif::[]
| If a User is associated with a Group in AD, keep the corresponding User as a member of the corresponding Group in Syncope.

| 
ifeval::["{snapshotOrRelease}" == "release"]
https://github.com/apache/syncope/blob/syncope-{docVersion}/core/provisioning-java/src/main/java/org/apache/syncope/core/provisioning/java/pushpull/LDAPMembershipPullActions.java[LDAPMembershipPullActions^]
endif::[]
ifeval::["{snapshotOrRelease}" == "snapshot"]
https://github.com/apache/syncope/blob/3_0_X/core/provisioning-java/src/main/java/org/apache/syncope/core/provisioning/java/pushpull/LDAPMembershipPullActions.java[LDAPMembershipPullActions^]
endif::[]
| If a User is associated with a Group in LDAP, keep the corresponding User as a member of the corresponding Group in Syncope.

| 
ifeval::["{snapshotOrRelease}" == "release"]
https://github.com/apache/syncope/blob/syncope-{docVersion}/core/provisioning-java/src/main/java/org/apache/syncope/core/provisioning/java/pushpull/LDAPPasswordPullActions.java[LDAPPasswordPullActions^]
endif::[]
ifeval::["{snapshotOrRelease}" == "snapshot"]
https://github.com/apache/syncope/blob/3_0_X/core/provisioning-java/src/main/java/org/apache/syncope/core/provisioning/java/pushpull/LDAPPasswordPullActions.java[LDAPPasswordPullActions^]
endif::[]
| Import hashed password values from LDAP;
the cipher algorithm associated with the password must match the value of `passwordHashAlgorithm` for the 
https://connid.atlassian.net/wiki/display/BASE/LDAP#LDAP-Configuration[LDAP connector bundle^].

| 
ifeval::["{snapshotOrRelease}" == "release"]
https://github.com/apache/syncope/blob/syncope-{docVersion}/core/provisioning-java/src/main/java/org/apache/syncope/core/provisioning/java/pushpull/DBPasswordPullActions.java[DBPasswordPullActions^]
endif::[]
ifeval::["{snapshotOrRelease}" == "snapshot"]
https://github.com/apache/syncope/blob/3_0_X/core/provisioning-java/src/main/java/org/apache/syncope/core/provisioning/java/pushpull/DBPasswordPullActions.java[DBPasswordPullActions^]
endif::[]
| Import hashed password values from DBMS;
the cipher algorithm associated with the password must match the value of `Password cipher algorithm` for the 
https://connid.atlassian.net/wiki/display/BASE/Database+Table#DatabaseTable-ConfigurationProperties[DatabaseTable connector bundle^].

|===

===== Remediation

Errors during pull might arise for various reasons: values might not be provided for all mandatory attributes or 
fail the configured validation, delete User as consequence of an incremental change's processing might be blocked
because such User is configured as Group owner, and so on.

When Remediation is enabled for a certain <<tasks-pull,Pull Task>>, execution errors are reported to administrators,
which are given the chance to examine and possibly fix, or just discard.
