0%

Elasticsearch 教程(3)文档CRUD操作

简介

Elasticsearch 是面向文档的,这就意味着它可以像MongoDB一样存储整个对象或者文档。然而它不仅仅是存储,还会索引每个文档的内容使值可以被索引。我们也可以对文档进行索引,搜索,排序,过滤。

它存储的文档格式是JSON格式的。比如:

1
{
2
    "email":      "john@smith.com",
3
    "first_name": "John",
4
    "last_name":  "Smith",
5
    "info": {
6
        "bio":         "Eco-warrior and defender of the weak",
7
        "age":         25,
8
        "interests": [ "dolphins", "whales" ]
9
    },
10
    "join_date": "2014/05/01"
11
}

理论

在Elasticsearch中存储数据的行为就叫做索引indexing)。文档属于一种类型type),而这些类型存储在索引index)中。

我们可以使用下面这个图来简单解释一下:

1
MySQL           ->  Database    ->  Tables      -> Row       -> Columns
2
MongoDB         ->  Database    ->  Collections -> Documents -> Fields
3
Elasticsearch   ->  Indices     ->  Types       -> Documents -> Fields

Elasticsearch集群可以包含多个索引(indices)(数据库),每一个索引可以包含多个类型(types)(表),每一个类型包含多个文档(documents)(行),然后每个文档包含多个段(Fields)(列)。

[索引]含义的区分

你可能已经注意到索引(index)这个词在Elasticsearch中有着不同的含义,所以有必要在此做一下区分:

  • 索引(名词) 如上文所述,一个索引(index)就像是传统关系数据库中的数据库,它是相关文档存储的地方,index的复数是indicesindexes
  • 索引(动词) 「索引一个文档」表示把一个文档存储到索引词)里,以便它可以被检索或者查询。这很像SQL中的INSERT关键字,差别是,如果文档已经存在,新的文档将覆盖旧的文档。
  • 倒排索引 传统数据库为特定列增加一个索引,例如B-Tree索引来加速检索。Elasticsearch和Lucene使用一种叫做倒排索引(inverted index)的数据结构来达到相同目的。

操作

插入文档

单个插入:

1
<?php
2
require_once './vendor/autoload.php';
3
4
$client = Elasticsearch\ClientBuilder::create();
5
$client->setHosts(['127.0.0.1']);
6
$client = $client->build();
7
8
$params = [
9
	'index' => 'megacorp', //索引
10
	'type' => 'employee', //类型
11
	'id' => 1, //ID 如果不加,则会自动生成
12
	'body' => [ //要添加的数据
13
		'first_name' => 'Join',
14
		'last_name' => 'Smith',
15
		'age' => 25,
16
		'about' => 'I love to go rock climbing',
17
		'interests' => ['sports', 'music'],
18
	]
19
];
20
$response = $client->index($params);
21
print_r($response);
22
?>

输出为:

1
Array
2
(
3
    [_index] => megacorp
4
    [_type] => employee
5
    [_id] => 1
6
    [_version] => 1
7
    [result] => created
8
    [_shards] => Array
9
        (
10
            [total] => 2
11
            [successful] => 1
12
            [failed] => 0
13
        )
14
    [created] => 1
15
)

就表示已经插入成功了。

这里的插入的数组中有个字段为id,如果我们不对其进行填写的话,它会自动生成一个id

这个自动生成的_id会有22个字符长。我们把它称作UUIDs

批量插入

下面我们接着插入。我们需要插入同时插入两个:

1
<?php
2
require_once './vendor/autoload.php';
3
$client = Elasticsearch\ClientBuilder::create();
4
$client->setHosts(['127.0.0.1']);
5
$client = $client->build();
6
7
$params = [];
8
$params['body'] = [
9
	[
10
		'index' => [
11
			'_index' => 'megacorp',
12
			'_type' => 'employee',
13
			'_id' => 2
14
		],
15
	],
16
	[
17
		"first_name" => "Jane",
18
	    "last_name" => "Smith",
19
	    "age" => 32,
20
	    "about" => "I like to collect rock albums",
21
	    "interests" => ["music"],
22
	],
23
	[
24
		'index' => [
25
			'_index' => 'megacorp',
26
			'_type' => 'employee',
27
			'_id' => 3
28
		],
29
	],
30
	[
31
		"first_name" => "Douglas",
32
	    "last_name" => "Fir",
33
	    "age" => 35,
34
	    "about" => "I like to build cabinets",
35
	    "interests" => [ "forestry" ]
36
	]
37
];
38
$responses = $client->bulk($params);
39
print_r($responses);
40
?>

显示如下就表示插入成功了:

1
Array
2
(
3
    [took] => 217
4
    [errors] => 
5
    [items] => Array
6
        (
7
            [0] => Array
8
                (
9
                    [index] => Array
10
                        (
11
                            [_index] => megacorp
12
                            [_type] => employee
13
                            [_id] => 2
14
                            [_version] => 1
15
                            [result] => created
16
                            [_shards] => Array
17
                                (
18
                                    [total] => 2
19
                                    [successful] => 1
20
                                    [failed] => 0
21
                                )
22
                            [created] => 1
23
                            [status] => 201
24
                        )
25
                )
26
            [1] => Array
27
                (
28
                    [index] => Array
29
                        (
30
                            [_index] => megacorp
31
                            [_type] => employee
32
                            [_id] => 3
33
                            [_version] => 1
34
                            [result] => created
35
                            [_shards] => Array
36
                                (
37
                                    [total] => 2
38
                                    [successful] => 1
39
                                    [failed] => 0
40
                                )
41
                            [created] => 1
42
                            [status] => 201
43
                        )
44
                )
45
        )
46
)

获取文档

现在我们尝试获取文档。获取文档可以获取指定文档的全部字段或者指定字段。我们分开来讲解:

获取单个文档

获取全部字段

比如我们现在要获取id=2的文档。

1
<?php
2
require_once './vendor/autoload.php';
3
$client = Elasticsearch\ClientBuilder::create();
4
$client->setHosts(['127.0.0.1']);
5
$client = $client->build();
6
7
$params = [
8
	'index' => 'megacorp', 
9
	'type' => 'employee',
10
	'id' => 2,
11
];
12
13
print_r($client->get($params));
14
?>

运行之后输出的结果就是:

1
Array
2
(
3
    [_index] => megacorp
4
    [_type] => employee
5
    [_id] => 2
6
    [_version] => 1
7
    [found] => 1
8
    [_source] => Array
9
        (
10
            [first_name] => Jane
11
            [last_name] => Smith
12
            [age] => 32
13
            [about] => I like to collect rock albums
14
            [interests] => Array
15
                (
16
                    [0] => music
17
                )
18
        )
19
)

这里我们可以看到_source字段包含的就是我们插入的内容。而found字段为1表示文档已经找到,如果我们请求一个不存在的文档,也会返回一个json,只不过found就会变成0了。

获取指定字段

比如我们这里用不到这么多的字段。我们仅仅需要first_name, last_nameage。我们可以这么请求:

1
<?php
2
require_once './vendor/autoload.php';
3
$client = Elasticsearch\ClientBuilder::create();
4
$client->setHosts(['127.0.0.1']);
5
$client = $client->build();
6
7
$params = [
8
	'index' => 'megacorp', 
9
	'type' => 'employee',
10
	'id' => 2,
11
	'_source' => ['first_name', 'last_name', 'age']
12
];
13
14
print_r($client->get($params));
15
?>

返回的结果仅仅是:

1
Array
2
(
3
    [_index] => megacorp
4
    [_type] => employee
5
    [_id] => 2
6
    [_version] => 1
7
    [found] => 1
8
    [_source] => Array
9
        (
10
            [last_name] => Smith
11
            [first_name] => Jane
12
            [age] => 32
13
        )
14
)

检查文档是否存在

如果我们不需要返回指定文档的内容,而仅仅是想知道文档是否存在,我们可以这样:

1
<?php
2
require_once './vendor/autoload.php';
3
$client = Elasticsearch\ClientBuilder::create();
4
$client->setHosts(['127.0.0.1']);
5
$client = $client->build();
6
7
$params = [
8
	'index' => 'megacorp', 
9
	'type' => 'employee',
10
	'id' => 2,
11
];
12
13
var_dump($client->exists($params));
14
?>

返回的结果不是数组了。而是一个bool值:

1
bool(true)

获取多个文档

获取全部的字段:

1
暂无

更新文档

部分文档更新

此处的更新只适合修改现有字段或者增加新的字段。我们需要在body字段中指定doc字段。
比如我说现在要修改id为2的员工。

首先我们先看看2号员工的信息:

1
Array
2
(
3
    [_index] => megacorp
4
    [_type] => employee
5
    [_id] => 2
6
    [_version] => 1
7
    [found] => 1
8
    [_source] => Array
9
        (
10
            [first_name] => Jane
11
            [last_name] => Smith
12
            [age] => 32
13
            [about] => I like to collect rock albums
14
            [interests] => Array
15
                (
16
                    [0] => music
17
                )
18
        )
19
)

接下来我们要修改它的信息,我们要将他的年龄修改成33,并且增加一个信息,mobile_phone1234567890

1
<?php
2
require_once './vendor/autoload.php';
3
$client = Elasticsearch\ClientBuilder::create();
4
$client->setHosts(['127.0.0.1']);
5
$client = $client->build();
6
7
$response = $client->update([
8
	'index' => 'megacorp',
9
	'type' => 'employee',
10
	'id' => 2,
11
	'body' => [
12
		'doc' => [
13
			'age' => 33,
14
			'mobile_phone' => '1234567890'
15
		]
16
	]
17
]);
18
print_r($response);
19
?>

结果如下:

1
Array
2
(
3
    [_index] => megacorp
4
    [_type] => employee
5
    [_id] => 2
6
    [_version] => 2
7
    [result] => updated
8
    [_shards] => Array
9
        (
10
            [total] => 2
11
            [successful] => 1
12
            [failed] => 0
13
        )
14
)

我们再来看一下之前的员工的信息:

1
Array
2
(
3
    [_index] => megacorp
4
    [_type] => employee
5
    [_id] => 2
6
    [_version] => 2
7
    [found] => 1
8
    [_source] => Array
9
        (
10
            [first_name] => Jane
11
            [last_name] => Smith
12
            [age] => 33
13
            [about] => I like to collect rock albums
14
            [interests] => Array
15
                (
16
                    [0] => music
17
                )
18
            [mobile_phone] => 1234567890
19
        )
20
)

脚本更新文档

有时候我们需要执行计数器更新,或者向数组中添加新值。我们就可以使用脚本式更新。

1
$params = [
2
	'index' => 'megacorp',
3
	'type' => 'employee',
4
	'id' => 2,
5
	'body' => [
6
		'script' => 'ctx._source.interests.add("sports")';
7
	]
8
]; // 数组中添加新值
9
10
$params = [
11
	'index' => 'megacorp',
12
	'type' => 'employee',
13
	'id' => 2,
14
	'body' => [
15
		'script' => 'ctx._source.age += 2';
16
	]
17
]; // 计数器更新

upsert

upsert 其实是更新或者插入操作,这意味着upsert将尝试更新操作,如果文档不存在,那么将插入默认值。

更新和冲突

为了避免更新数据,updateAPI在解锁阶段检索文档当前的_version,然后在重建索引阶段通过index请求提交,如果其它进程在检索和重建索引阶段修改了文档,_version将不能被匹配,然后更新失败。

对于这种情况,我们只需要重新尝试更新就好了,其实这些我们可以通过retry_on_conflict参数设置重试次数来自动完成,这样update操作将会在发生错误前重试——这个值默认为0。

总结

其实update这个操作似乎允许你修改文档的局部,但实际上还是遵循先查后改的过程,步骤如下:

  1. 从旧文档中检索JSON
  2. 修改它
  3. 删除旧文档
  4. 索引新文档

唯一的不同是update这个操作只需要一个客户端请求就好,不需要getindex请求了。

删除

比如我们要删除id为3的员工:

我们先查询一下这个员工:

1
Array
2
(
3
    [_index] => megacorp
4
    [_type] => employee
5
    [_id] => 3
6
    [_version] => 1
7
    [found] => 1
8
    [_source] => Array
9
        (
10
            [first_name] => Douglas
11
            [last_name] => Fir
12
            [age] => 35
13
            [about] => I like to build cabinets
14
            [interests] => Array
15
                (
16
                    [0] => forestry
17
                )
18
        )
19
)

下面我们来执行操作:

1
print_r($client->delete([
2
	'index' => 'megacorp',
3
	'type' => 'employee',
4
	'id' => 3,
5
]));

返回结果为:

1
Array
2
(
3
    [found] => 1
4
    [_index] => megacorp
5
    [_type] => employee
6
    [_id] => 3
7
    [_version] => 2
8
    [result] => deleted
9
    [_shards] => Array
10
        (
11
            [total] => 2
12
            [successful] => 1
13
            [failed] => 0
14
        )
15
)

注意看found为1,并且_version相比之前,已经变为2了。

当我们再次执行一下之前的删除操作,我们再看一下返回结果:

1
{"found":false,"_index":"megacorp","_type":"employee","_id":"3","_version":1,"result":"not_found","_shards":{"total":2,"successful":1,"failed":0}}
1
{
2
  "found": false,
3
  "_index": "megacorp",
4
  "_type": "employee",
5
  "_id": "3",
6
  "_version": 2,
7
  "result": "not_found",
8
  "_shards": {
9
    "total": 2,
10
    "successful": 1,
11
    "failed": 0
12
  }
13
}

删除不存在的文档的时候,抛出了一个错误。我们可以看到found的值是false,且_version也有记录值。这是内部记录的一部分,它确保再多节点不同操作可以有正确的顺序。