1. ES的Mapping设计原则?keyword和text的区别?
2. ik_smart和ik_max_word的区别?索引和搜索时各用哪个?
3. bool查询中must、should、filter的区别?
4. 聚合查询怎么写?嵌套聚合和Pipeline聚合?
5. 热温冷架构怎么设计?ILM策略怎么配?
6. ES处理更新的原理?为什么不适合频繁更新?
7. 慢查询怎么排查和优化?
PUT /product
{
"settings": {
"number_of_shards": 3,
"number_of_replicas": 1,
"analysis": {
"analyzer": {
"ik_smart_analyzer": { "type": "custom", "tokenizer": "ik_smart" },
"ik_max_analyzer": { "type": "custom", "tokenizer": "ik_max_word" },
"pinyin_analyzer": { "type": "custom", "tokenizer": "pinyin" }
}
}
},
"mappings": {
"properties": {
"productId": { "type": "long" },
"productName": {
"type": "text", "analyzer": "ik_max_word", "search_analyzer": "ik_smart",
"fields": { "keyword": { "type": "keyword" }, "pinyin": { "type": "text", "analyzer": "pinyin_analyzer" } }
},
"brandName": { "type": "keyword" },
"categoryName": { "type": "keyword", "fields": { "text": { "type": "text", "analyzer": "ik_smart" } } },
"price": { "type": "scaled_float", "scaling_factor": 100 },
"sales": { "type": "integer" },
"status": { "type": "integer" },
"description": { "type": "text", "analyzer": "ik_max_word", "search_analyzer": "ik_smart" },
"tags": { "type": "keyword" },
"createTime": { "type": "date", "format": "yyyy-MM-dd HH:mm:ss||epoch_millis" }
}
}
}
| 类型 | 适用场景 | 分词 | 聚合 |
|---|---|---|---|
| keyword | 精确匹配/排序/聚合 | 否 | 是 |
| text | 全文搜索 | 是 | 否(需fielddata) |
| scaled_float | 价格等浮点 | 否 | 是 |
| nested | 对象数组独立查询 | 按字段 | 按字段 |
ik_max_word(索引时):最细粒度切分 → 召回率高
"中华人民共和国" → "中华人民共和国","中华人民","中华","华人","人民共和国",...
ik_smart(搜索时):最粗粒度切分 → 准确率高
"中华人民共和国" → "中华人民共和国"
设计原则:索引时最大分词 + 搜索时智能分词 = 召回率与准确率的平衡
standard(默认):英文按词切分,中文按单字切分(效果差)
写入流程:
1. 客户端 → 协调节点 → routing计算目标分片
2. 主分片:写Index Buffer + Translog → 每隔refresh_interval(1s)刷新为Segment(可搜索)
3. 并行转发副本 → 等待确认 → 返回成功
4. 后台flush:Segment持久化 + 清理Translog
更新原理(没有原地更新):
1. 标记旧文档为deleted(逻辑删除)
2. 写入全新文档(新版本号)
3. 后台Segment Merge时物理删除
结论:每次更新 = 标记删除 + 重新写入,开销约为新增的2倍
不适合频繁更新场景(库存扣减/计数器 → 用Redis)
GET /product/_search
{
"query": {
"bool": {
"must": [
{ "multi_match": {
"query": "华为手机",
"fields": ["productName^3", "description"],
"type": "best_fields",
"fuzziness": "AUTO"
}}
],
"filter": [
{ "term": { "brandName": "华为" } },
{ "range": { "price": { "gte": 2000, "lte": 6000 } } },
{ "term": { "status": 1 } }
],
"should": [
{ "term": { "categoryName": { "value": "智能手机", "boost": 2 } } }
],
"minimum_should_match": 0
}
},
"sort": [ { "_score": "desc" }, { "sales": "desc" } ],
"from": 0, "size": 20,
"highlight": { "fields": { "productName": { "pre_tags": ["<em>"], "post_tags": ["</em>"] } } }
}
must:必须匹配,参与评分,用于相关性搜索
filter:必须匹配,不参与评分,结果被位图缓存,性能最高
should:可选匹配,参与评分
must_not:必须不匹配,不参与评分
性能:filter > must_not > should > must
默认:每次查询命中所有分片 → 汇总排序
自定义routing:查询只命中目标分片,减少N倍IO
写入时:PUT /product/_doc/1?routing=huawei
查询时:GET /product/_search?routing=huawei
注意:数据均匀字段才适合做routing,基数太低(status)会导致倾斜
Terms + Range聚合(搜索页筛选面板)
GET /product/_search
{
"query": { "match": { "productName": "手机" } },
"size": 0,
"aggs": {
"brand_distribution": {
"terms": { "field": "brandName", "size": 20 }
},
"price_ranges": {
"range": { "field": "price",
"ranges": [
{ "key": "0-1000", "to": 1000 },
{ "key": "1000-3000", "from": 1000, "to": 3000 },
{ "key": "3000-5000", "from": 3000, "to": 5000 },
{ "key": "5000+", "from": 5000 }
]
}
},
"price_stats": { "stats": { "field": "price" } }
}
}
嵌套聚合(品牌 → 价格区间 → 平均销量)
GET /product/_search
{
"size": 0,
"aggs": {
"by_brand": {
"terms": { "field": "brandName", "size": 10 },
"aggs": {
"by_price_range": {
"range": { "field": "price",
"ranges": [
{ "key": "低端", "to": 1500 },
{ "key": "中端", "from": 1500, "to": 4000 },
{ "key": "高端", "from": 4000 }
]
},
"aggs": { "avg_sales": { "avg": { "field": "sales" } } }
},
"brand_avg_price": { "avg": { "field": "price" } }
}
}
}
}
日期直方图(按天订单趋势)
GET /order/_search
{
"size": 0,
"query": { "range": { "createTime": { "gte": "now-30d" } } },
"aggs": {
"daily_orders": {
"date_histogram": {
"field": "createTime",
"calendar_interval": "day",
"format": "yyyy-MM-dd",
"min_doc_count": 0,
"extended_bounds": { "min": "now-30d", "max": "now" }
},
"aggs": {
"total_amount": { "sum": { "field": "amount" } },
"avg_amount": { "avg": { "field": "amount" } }
}
}
}
}
Pipeline聚合(日环比增长率)
GET /order/_search
{
"size": 0,
"aggs": {
"daily": {
"date_histogram": { "field": "createTime", "calendar_interval": "day" },
"aggs": { "amount": { "sum": { "field": "amount" } } }
},
"growth_rate": {
"derivative": { "buckets_path": "daily>amount" }
}
}
}
@Document(indexName = "product")
@Setting(shards = 3, replicas = 1)
public class ProductDocument {
@Id
private Long productId;
@Field(type = FieldType.Text, analyzer = "ik_max_word", searchAnalyzer = "ik_smart")
private String productName;
@Field(type = FieldType.Keyword)
private String brandName;
@Field(type = FieldType.Keyword)
private String categoryName;
@Field(type = FieldType.Double)
private Double price;
@Field(type = FieldType.Integer)
private Integer sales;
@Field(type = FieldType.Integer)
private Integer status;
@Field(type = FieldType.Text, analyzer = "ik_max_word", searchAnalyzer = "ik_smart")
private String description;
@Field(type = FieldType.Date, format = DateFormat.date_hour_minute_second)
private LocalDateTime createTime;
// getter/setter ...
}
@Service
@Slf4j
public class ProductSearchService {
@Autowired private ElasticsearchOperations esOps;
@Autowired private ProductRepository productRepo;
/**
* 商品搜索:关键词 + 品牌 + 价格区间 + 聚合
*/
public SearchPage<ProductDocument> search(String keyword, String brand,
Double minPrice, Double maxPrice, int pageNum, int pageSize) {
BoolQueryBuilder boolQuery = QueryBuilders.boolQuery();
// 关键词 → must(影响评分)
if (StringUtils.hasText(keyword)) {
boolQuery.must(QueryBuilders.multiMatchQuery(keyword,
"productName", "description")
.type(MultiMatchQueryBuilder.Type.BEST_FIELDS)
.fuzziness(Fuzziness.AUTO));
}
// 精确过滤 → filter(不影响评分,利用缓存)
if (StringUtils.hasText(brand))
boolQuery.filter(QueryBuilders.termQuery("brandName", brand));
if (minPrice != null && maxPrice != null)
boolQuery.filter(QueryBuilders.rangeQuery("price").gte(minPrice).lte(maxPrice));
boolQuery.filter(QueryBuilders.termQuery("status", 1));
NativeSearchQueryBuilder qb = new NativeSearchQueryBuilder()
.withQuery(boolQuery)
.withPageable(PageRequest.of(pageNum, pageSize))
.withSort(SortBuilders.scoreSort().order(SortOrder.DESC))
.withSort(SortBuilders.fieldSort("sales").order(SortOrder.DESC));
// 品牌聚合 + 价格区间聚合(搜索页筛选面板)
qb.addAggregation(AggregationBuilders.terms("brand_agg").field("brandName").size(20));
qb.addAggregation(AggregationBuilders.range("price_agg").field("price")
.addRange("0-1000", 0, 1000).addRange("1000-3000", 1000, 3000)
.addRange("3000-5000", 3000, 5000).addRange("5000+", 5000, null));
// 高亮
qb.withHighlightBuilder(new HighlightBuilder()
.field("productName").preTags("<em>").postTags("</em>"));
SearchHits<ProductDocument> hits = esOps.search(qb.build(), ProductDocument.class);
return SearchHitSupport.searchPageFor(hits, PageRequest.of(pageNum, pageSize));
}
/** 增量同步:MySQL → ES */
public void incrementalSync(LocalDateTime since) {
List<Product> products = productMapper.selectList(
new LambdaQueryWrapper<Product>().ge(Product::getUpdateTime, since));
productRepo.saveAll(products.stream().map(this::toDoc).collect(Collectors.toList()));
}
}
热温冷架构 + ILM
Hot(SSD):索引写入 + 热数据查询 node.roles: [data_hot, data_content]
Warm(HDD):只查询不索引 node.roles: [data_warm]
Cold(大容量HDD):冻结索引 node.roles: [data_cold]
PUT /_ilm/policy/product-policy
{
"policy": {
"phases": {
"hot": {
"min_age": "0ms",
"actions": {
"rollover": { "max_size": "50gb", "max_age": "7d" },
"set_priority": { "priority": 100 }
}
},
"warm": {
"min_age": "7d",
"actions": {
"shrink": { "number_of_shards": 1 },
"forcemerge": { "max_num_segments": 1 },
"allocate": { "require": { "data": "warm" } }
}
},
"cold": {
"min_age": "30d",
"actions": {
"freeze": {},
"allocate": { "require": { "data": "cold" } }
}
},
"delete": {
"min_age": "90d",
"actions": { "delete": {} }
}
}
}
}
// 绑定ILM到索引模板
PUT /_index_template/product-template
{
"index_patterns": ["product-*"],
"template": {
"settings": {
"number_of_shards": 3,
"number_of_replicas": 1,
"lifecycle.name": "product-policy",
"lifecycle.rollover_alias": "product"
}
}
}
Reindex(索引重建 + 零停机切换)
// 1. 重建索引(可带script转换字段)
POST /_reindex
{
"source": { "index": "product-v1" },
"dest": { "index": "product-v2" }
}
// 2. 原子切换别名
POST /_aliases
{
"actions": [
{ "remove": { "index": "product-v1", "alias": "product" } },
{ "add": { "index": "product-v2", "alias": "product" } }
]
}
Q1:keyword和text的区别?
keyword不分词,整体存储,适合精确匹配/排序/聚合。text经分词器处理,适合全文搜索。多字段(fields)可让一个字段同时拥有两种类型。
Q2:filter和must的区别?
must参与评分计算,filter不参与。filter结果被位图缓存,第二次直接命中缓存,性能远高于must。精确过滤(品牌/状态/价格)全部放filter。
Q3:ES处理更新的原理?
没有原地更新:标记旧文档deleted → 写入全新文档 → 后台Merge物理删除。每次更新开销约为新增的2倍,不适合频繁更新(库存/计数器用Redis)。
Q4:深度分页怎么解决?
from+size深度分页:每分片返回from+size条,协调节点汇总排序取size条,性能极差。解决:search_after(实时翻页,基于上一页排序值)、Scroll API(批量导出)、业务限制最大页码。
Q5:慢查询优化?
开启慢查询日志(slowlog.threshold.query.warn: 5s) → Profile分析(?profile=true) → 检查:filter未用keyword、深度分页、通配符查询、高基数聚合 → 优化:精确过滤放filter、routing减少分片、预索引字段、冷热分离。
练习1:索引与分词器验证
1. 创建商品索引(使用上方1.1 Mapping)
2. 插入测试数据
3. _analyze API对比 ik_max_word vs ik_smart 分词效果
练习2:查询与聚合
1. 实现关键词搜索 + 品牌filter + 价格filter
2. 添加品牌terms聚合 + 价格range聚合(筛选面板)
3. 嵌套聚合:品牌 → 价格区间 → 平均销量
4. 对比 filter vs must 查询耗时
练习3:ILM与Reindex
1. 创建ILM策略:热(1天) → 温(3天) → 删除(7天)
2. 写入测试数据,观察rollover
3. Reindex重建索引 + 别名原子切换
| 知识点 | 关键内容 |
|---|---|
| Mapping设计 | keyword vs text、multi-field、scaled_float |
| 分词器 | ik_max_word(索引/召回率) + ik_smart(搜索/准确率) |
| 查询优化 | bool查询、filter缓存、routing、避免深度分页 |
| 聚合查询 | terms、range、date_histogram、嵌套聚合、pipeline |
| 生产实践 | 热温冷架构、ILM、Reindex+别名切换 |
| 更新原理 | 标记删除+重写,不适合频繁更新 |