← 返回首页 📚 实战课程
📑 本篇章节导航 (共6章,点击直达)
本节目标(学完能回答的面试题)一、理论篇(30%):索引设计与查询原理二、实战篇(40%):查询优化与Spring Data Elasticsearch三、面试模拟篇(20%)四、练习篇(10%)本节小结

P2-06-Elasticsearch进阶与生产实践

本节目标(学完能回答的面试题)

1. ES的Mapping设计原则?keyword和text的区别?

2. ik_smart和ik_max_word的区别?索引和搜索时各用哪个?

3. bool查询中must、should、filter的区别?

4. 聚合查询怎么写?嵌套聚合和Pipeline聚合?

5. 热温冷架构怎么设计?ILM策略怎么配?

6. ES处理更新的原理?为什么不适合频繁更新?

7. 慢查询怎么排查和优化?


一、理论篇(30%):索引设计与查询原理

1.1 Mapping设计


PUT /product
{
  "settings": {
    "number_of_shards": 3,
    "number_of_replicas": 1,
    "analysis": {
      "analyzer": {
        "ik_smart_analyzer": { "type": "custom", "tokenizer": "ik_smart" },
        "ik_max_analyzer":   { "type": "custom", "tokenizer": "ik_max_word" },
        "pinyin_analyzer":   { "type": "custom", "tokenizer": "pinyin" }
      }
    }
  },
  "mappings": {
    "properties": {
      "productId":    { "type": "long" },
      "productName":  {
        "type": "text", "analyzer": "ik_max_word", "search_analyzer": "ik_smart",
        "fields": { "keyword": { "type": "keyword" }, "pinyin": { "type": "text", "analyzer": "pinyin_analyzer" } }
      },
      "brandName":    { "type": "keyword" },
      "categoryName": { "type": "keyword", "fields": { "text": { "type": "text", "analyzer": "ik_smart" } } },
      "price":        { "type": "scaled_float", "scaling_factor": 100 },
      "sales":        { "type": "integer" },
      "status":       { "type": "integer" },
      "description":  { "type": "text", "analyzer": "ik_max_word", "search_analyzer": "ik_smart" },
      "tags":         { "type": "keyword" },
      "createTime":   { "type": "date", "format": "yyyy-MM-dd HH:mm:ss||epoch_millis" }
    }
  }
}

1.2 字段类型与分词器选择

类型适用场景分词聚合
keyword精确匹配/排序/聚合否是
text全文搜索是否(需fielddata)
scaled_float价格等浮点否是
nested对象数组独立查询按字段按字段

ik_max_word(索引时):最细粒度切分 → 召回率高
  "中华人民共和国" → "中华人民共和国","中华人民","中华","华人","人民共和国",...

ik_smart(搜索时):最粗粒度切分 → 准确率高
  "中华人民共和国" → "中华人民共和国"

设计原则:索引时最大分词 + 搜索时智能分词 = 召回率与准确率的平衡

standard(默认):英文按词切分,中文按单字切分(效果差)

1.3 ES写入与更新原理


写入流程:
1. 客户端 → 协调节点 → routing计算目标分片
2. 主分片:写Index Buffer + Translog → 每隔refresh_interval(1s)刷新为Segment(可搜索)
3. 并行转发副本 → 等待确认 → 返回成功
4. 后台flush:Segment持久化 + 清理Translog

更新原理(没有原地更新):
1. 标记旧文档为deleted(逻辑删除)
2. 写入全新文档(新版本号)
3. 后台Segment Merge时物理删除

结论:每次更新 = 标记删除 + 重新写入,开销约为新增的2倍
     不适合频繁更新场景(库存扣减/计数器 → 用Redis)

二、实战篇(40%):查询优化与Spring Data Elasticsearch

2.1 Bool查询


GET /product/_search
{
  "query": {
    "bool": {
      "must": [
        { "multi_match": {
            "query": "华为手机",
            "fields": ["productName^3", "description"],
            "type": "best_fields",
            "fuzziness": "AUTO"
        }}
      ],
      "filter": [
        { "term":  { "brandName": "华为" } },
        { "range": { "price": { "gte": 2000, "lte": 6000 } } },
        { "term":  { "status": 1 } }
      ],
      "should": [
        { "term": { "categoryName": { "value": "智能手机", "boost": 2 } } }
      ],
      "minimum_should_match": 0
    }
  },
  "sort": [ { "_score": "desc" }, { "sales": "desc" } ],
  "from": 0, "size": 20,
  "highlight": { "fields": { "productName": { "pre_tags": ["<em>"], "post_tags": ["</em>"] } } }
}

must:必须匹配,参与评分,用于相关性搜索
filter:必须匹配,不参与评分,结果被位图缓存,性能最高
should:可选匹配,参与评分
must_not:必须不匹配,不参与评分
性能:filter > must_not > should > must

2.2 Routing优化


默认:每次查询命中所有分片 → 汇总排序
自定义routing:查询只命中目标分片,减少N倍IO

写入时:PUT /product/_doc/1?routing=huawei
查询时:GET /product/_search?routing=huawei

注意:数据均匀字段才适合做routing,基数太低(status)会导致倾斜

2.3 聚合查询

Terms + Range聚合(搜索页筛选面板)


GET /product/_search
{
  "query": { "match": { "productName": "手机" } },
  "size": 0,
  "aggs": {
    "brand_distribution": {
      "terms": { "field": "brandName", "size": 20 }
    },
    "price_ranges": {
      "range": { "field": "price",
        "ranges": [
          { "key": "0-1000", "to": 1000 },
          { "key": "1000-3000", "from": 1000, "to": 3000 },
          { "key": "3000-5000", "from": 3000, "to": 5000 },
          { "key": "5000+", "from": 5000 }
        ]
      }
    },
    "price_stats": { "stats": { "field": "price" } }
  }
}

嵌套聚合(品牌 → 价格区间 → 平均销量)


GET /product/_search
{
  "size": 0,
  "aggs": {
    "by_brand": {
      "terms": { "field": "brandName", "size": 10 },
      "aggs": {
        "by_price_range": {
          "range": { "field": "price",
            "ranges": [
              { "key": "低端", "to": 1500 },
              { "key": "中端", "from": 1500, "to": 4000 },
              { "key": "高端", "from": 4000 }
            ]
          },
          "aggs": { "avg_sales": { "avg": { "field": "sales" } } }
        },
        "brand_avg_price": { "avg": { "field": "price" } }
      }
    }
  }
}

日期直方图(按天订单趋势)


GET /order/_search
{
  "size": 0,
  "query": { "range": { "createTime": { "gte": "now-30d" } } },
  "aggs": {
    "daily_orders": {
      "date_histogram": {
        "field": "createTime",
        "calendar_interval": "day",
        "format": "yyyy-MM-dd",
        "min_doc_count": 0,
        "extended_bounds": { "min": "now-30d", "max": "now" }
      },
      "aggs": {
        "total_amount": { "sum": { "field": "amount" } },
        "avg_amount":   { "avg": { "field": "amount" } }
      }
    }
  }
}

Pipeline聚合(日环比增长率)


GET /order/_search
{
  "size": 0,
  "aggs": {
    "daily": {
      "date_histogram": { "field": "createTime", "calendar_interval": "day" },
      "aggs": { "amount": { "sum": { "field": "amount" } } }
    },
    "growth_rate": {
      "derivative": { "buckets_path": "daily>amount" }
    }
  }
}

2.4 Spring Data Elasticsearch


@Document(indexName = "product")
@Setting(shards = 3, replicas = 1)
public class ProductDocument {
    @Id
    private Long productId;
    @Field(type = FieldType.Text, analyzer = "ik_max_word", searchAnalyzer = "ik_smart")
    private String productName;
    @Field(type = FieldType.Keyword)
    private String brandName;
    @Field(type = FieldType.Keyword)
    private String categoryName;
    @Field(type = FieldType.Double)
    private Double price;
    @Field(type = FieldType.Integer)
    private Integer sales;
    @Field(type = FieldType.Integer)
    private Integer status;
    @Field(type = FieldType.Text, analyzer = "ik_max_word", searchAnalyzer = "ik_smart")
    private String description;
    @Field(type = FieldType.Date, format = DateFormat.date_hour_minute_second)
    private LocalDateTime createTime;
    // getter/setter ...
}

@Service
@Slf4j
public class ProductSearchService {
    @Autowired private ElasticsearchOperations esOps;
    @Autowired private ProductRepository productRepo;

    /**
     * 商品搜索:关键词 + 品牌 + 价格区间 + 聚合
     */
    public SearchPage<ProductDocument> search(String keyword, String brand,
            Double minPrice, Double maxPrice, int pageNum, int pageSize) {

        BoolQueryBuilder boolQuery = QueryBuilders.boolQuery();

        // 关键词 → must(影响评分)
        if (StringUtils.hasText(keyword)) {
            boolQuery.must(QueryBuilders.multiMatchQuery(keyword,
                "productName", "description")
                .type(MultiMatchQueryBuilder.Type.BEST_FIELDS)
                .fuzziness(Fuzziness.AUTO));
        }

        // 精确过滤 → filter(不影响评分,利用缓存)
        if (StringUtils.hasText(brand))
            boolQuery.filter(QueryBuilders.termQuery("brandName", brand));
        if (minPrice != null && maxPrice != null)
            boolQuery.filter(QueryBuilders.rangeQuery("price").gte(minPrice).lte(maxPrice));
        boolQuery.filter(QueryBuilders.termQuery("status", 1));

        NativeSearchQueryBuilder qb = new NativeSearchQueryBuilder()
            .withQuery(boolQuery)
            .withPageable(PageRequest.of(pageNum, pageSize))
            .withSort(SortBuilders.scoreSort().order(SortOrder.DESC))
            .withSort(SortBuilders.fieldSort("sales").order(SortOrder.DESC));

        // 品牌聚合 + 价格区间聚合(搜索页筛选面板)
        qb.addAggregation(AggregationBuilders.terms("brand_agg").field("brandName").size(20));
        qb.addAggregation(AggregationBuilders.range("price_agg").field("price")
            .addRange("0-1000", 0, 1000).addRange("1000-3000", 1000, 3000)
            .addRange("3000-5000", 3000, 5000).addRange("5000+", 5000, null));

        // 高亮
        qb.withHighlightBuilder(new HighlightBuilder()
            .field("productName").preTags("<em>").postTags("</em>"));

        SearchHits<ProductDocument> hits = esOps.search(qb.build(), ProductDocument.class);
        return SearchHitSupport.searchPageFor(hits, PageRequest.of(pageNum, pageSize));
    }

    /** 增量同步:MySQL → ES */
    public void incrementalSync(LocalDateTime since) {
        List<Product> products = productMapper.selectList(
            new LambdaQueryWrapper<Product>().ge(Product::getUpdateTime, since));
        productRepo.saveAll(products.stream().map(this::toDoc).collect(Collectors.toList()));
    }
}

2.5 生产实践

热温冷架构 + ILM


Hot(SSD):索引写入 + 热数据查询   node.roles: [data_hot, data_content]
Warm(HDD):只查询不索引          node.roles: [data_warm]
Cold(大容量HDD):冻结索引         node.roles: [data_cold]

PUT /_ilm/policy/product-policy
{
  "policy": {
    "phases": {
      "hot": {
        "min_age": "0ms",
        "actions": {
          "rollover": { "max_size": "50gb", "max_age": "7d" },
          "set_priority": { "priority": 100 }
        }
      },
      "warm": {
        "min_age": "7d",
        "actions": {
          "shrink": { "number_of_shards": 1 },
          "forcemerge": { "max_num_segments": 1 },
          "allocate": { "require": { "data": "warm" } }
        }
      },
      "cold": {
        "min_age": "30d",
        "actions": {
          "freeze": {},
          "allocate": { "require": { "data": "cold" } }
        }
      },
      "delete": {
        "min_age": "90d",
        "actions": { "delete": {} }
      }
    }
  }
}

// 绑定ILM到索引模板
PUT /_index_template/product-template
{
  "index_patterns": ["product-*"],
  "template": {
    "settings": {
      "number_of_shards": 3,
      "number_of_replicas": 1,
      "lifecycle.name": "product-policy",
      "lifecycle.rollover_alias": "product"
    }
  }
}

Reindex(索引重建 + 零停机切换)


// 1. 重建索引(可带script转换字段)
POST /_reindex
{
  "source": { "index": "product-v1" },
  "dest":   { "index": "product-v2" }
}

// 2. 原子切换别名
POST /_aliases
{
  "actions": [
    { "remove": { "index": "product-v1", "alias": "product" } },
    { "add":    { "index": "product-v2", "alias": "product" } }
  ]
}

三、面试模拟篇(20%)

Q1:keyword和text的区别?

keyword不分词,整体存储,适合精确匹配/排序/聚合。text经分词器处理,适合全文搜索。多字段(fields)可让一个字段同时拥有两种类型。

Q2:filter和must的区别?

must参与评分计算,filter不参与。filter结果被位图缓存,第二次直接命中缓存,性能远高于must。精确过滤(品牌/状态/价格)全部放filter。

Q3:ES处理更新的原理?

没有原地更新:标记旧文档deleted → 写入全新文档 → 后台Merge物理删除。每次更新开销约为新增的2倍,不适合频繁更新(库存/计数器用Redis)。

Q4:深度分页怎么解决?

from+size深度分页:每分片返回from+size条,协调节点汇总排序取size条,性能极差。解决:search_after(实时翻页,基于上一页排序值)、Scroll API(批量导出)、业务限制最大页码。

Q5:慢查询优化?

开启慢查询日志(slowlog.threshold.query.warn: 5s) → Profile分析(?profile=true) → 检查:filter未用keyword、深度分页、通配符查询、高基数聚合 → 优化:精确过滤放filter、routing减少分片、预索引字段、冷热分离。

四、练习篇(10%)


练习1:索引与分词器验证
  1. 创建商品索引(使用上方1.1 Mapping)
  2. 插入测试数据
  3. _analyze API对比 ik_max_word vs ik_smart 分词效果

练习2:查询与聚合
  1. 实现关键词搜索 + 品牌filter + 价格filter
  2. 添加品牌terms聚合 + 价格range聚合(筛选面板)
  3. 嵌套聚合:品牌 → 价格区间 → 平均销量
  4. 对比 filter vs must 查询耗时

练习3:ILM与Reindex
  1. 创建ILM策略:热(1天) → 温(3天) → 删除(7天)
  2. 写入测试数据,观察rollover
  3. Reindex重建索引 + 别名原子切换

本节小结

知识点关键内容
Mapping设计keyword vs text、multi-field、scaled_float
分词器ik_max_word(索引/召回率) + ik_smart(搜索/准确率)
查询优化bool查询、filter缓存、routing、避免深度分页
聚合查询terms、range、date_histogram、嵌套聚合、pipeline
生产实践热温冷架构、ILM、Reindex+别名切换
更新原理标记删除+重写,不适合频繁更新