Getting set up

Start an RStudio session at OSC

Show instructions

  • Fill out the form as shown here.

  • Now, you should see a box like this:

  • Your job should start running pretty soon, and when it’s ready the box should look like this:

  • Click Connect to RStudio Server at the bottom of the box, and an RStudio Server instance will open. You’re ready to go!

Create an RStudio Project

Show instructions
basedir <- "/fs/project/PAS0471/teach/misc/2021-02_rnaseq/"

# Get your user name (by running a shell command via the `system()` function:
me <- system("echo $USER", intern = TRUE)

# Build the path to the target dir:
proj_dir <- file.path(basedir, me)

# Create the Project:
library(usethis)
create_project(path = proj_dir)

Now, RStudio will reload with the newly created Project open.

If you get the pop-up below, click Don't Save (do this whenever you get that pop-up):

Copy the count data and the metadata files

Go the Terminal tab next to the R Console in the bottom left of the RStudio window:

cd /fs/project/PAS0471/teach/misc/2021-02_rnaseq/$USER

cp ../master/data/meta/metadata.txt data/meta
cp ../master/results/count/gene_counts_all.txt results/count

Load the necessary packages

if(! "pacman" %in% installed.packages()) install.packages("pacman")

packages <- c("DESeq2",          # Differential expression analysis
              "tidyverse",       # Misc. data manipulation and plotting
              "here",            # Managing file paths
              "pheatmap",        # Heatmap plot
              "apeglm",          # For LFC shrinkage
              "knitr")           # For table printing in Markdown file
pacman::p_load(char = packages)

theme_set(theme_bw())  # Set ggplot theme

Input and output dirs and files

Input files:

  • Gene counts table – the file exactly as it was written by featureCounts.

  • Metadata table – so we can group our samples and make comparisons between these groups.

count_table_file <- here("results/count/gene_counts_all.txt")
metadata_file <- here("data/meta/metadata.txt")

Output directories – and we create them if they don’t already exist:

outdir <- here("results/DE/")
plotdir <- here("results/DE/fig/")

if (!dir.exists(plotdir)) dir.create(plotdir, recursive = TRUE)
if (!dir.exists(plotdir)) dir.create(plotdir, recursive = TRUE)

Load input data

Load the count table from featureCounts:

raw_counts <- read.table(count_table_file,
                         sep = "\t", header = TRUE, skip = 1)

Load the metadata information:

metadata <- read.table(metadata_file, header = TRUE)

kable(head(metadata))
SampleID Experiment GH_trial AMF Treatment
C6_myb2 CAPS CAPS_6 Ri Agrobacterium_noexp
C6_myb3 CAPS CAPS_6 Ri Agrobacterium_noexp
C6_myb4 CAPS CAPS_6 Ri Agrobacterium_noexp
CT_05 CAPS CAPS_6 Ri mock
CT_06 CAPS CAPS_6 Ri mock
CT_09 CAPS CAPS_6 Ri mock

The Treatment column currently has the values Agrobacterium_noexp, Agrobacterium_myb, and mock.

To shorten this a bit, we’ll get rid of “Agrobacterium_”:

metadata$Treatment <- sub("Agrobacterium_", "", metadata$Treatment)

unique(metadata$Treatment)
## [1] "noexp" "mock"  "myb"



Prepare the data

Change the column names, which are very long:

colnames(raw_counts)[7:8]
## [1] "X.fs.scratch.PAS1548.JamesSeq.CAPS.tomato.star_out.X6465_Benitez.PonceM_C6_myb2_V1N_1_S1_L001_R1_001.fastq.gzAligned.sortedByCoord.out.bam"
## [2] "X.fs.scratch.PAS1548.JamesSeq.CAPS.tomato.star_out.X6465_Benitez.PonceM_C6_myb2_V1N_1_S1_L002_R1_001.fastq.gzAligned.sortedByCoord.out.bam"
my_regex <- ".+PonceM_(.+)_V1N.+"
colnames(raw_counts) <- sub(my_regex, "\\1", colnames(raw_counts))

colnames(raw_counts)
##  [1] "Geneid"  "Chr"     "Start"   "End"     "Strand"  "Length"  "C6_myb2"
##  [8] "C6_myb2" "C6_myb3" "C6_myb3" "C6_myb4" "C6_myb4" "CT_05"   "CT_05"  
## [15] "CT_06"   "CT_06"   "CT_09"   "CT_09"   "T_26"    "T_26"    "T_29"   
## [22] "T_29"    "T_30"    "T_30"    "cnt_1"   "cnt_1"   "cnt_2"   "cnt_2"  
## [29] "cnt_3"   "cnt_3"   "myb_1"   "myb_1"   "myb_2"   "myb_2"   "myb_3"  
## [36] "myb_3"

Besides the counts, there are columns with metadata for each gene:

raw_counts[1:5, 1:8]
##             Geneid       Chr  Start    End Strand Length C6_myb2 C6_myb2.1
## 1 Solyc00g160260.1 SL4.0ch00  83863  84177      +    315       0         0
## 2 Solyc00g160270.1 SL4.0ch00 166754 167268      -    515       0         0
## 3 Solyc00g500003.1 SL4.0ch00 311496 382066      -  70571       1         4
## 4 Solyc00g500004.1 SL4.0ch00 417592 418482      +    891       0         0
## 5 Solyc00g500005.1 SL4.0ch00 478389 478640      +    252       0         0

Let’s remove those:

counts <- raw_counts[, 7:ncol(raw_counts)]
rownames(counts) <- raw_counts$Geneid

Replicate samples

In this table, there are two separate entries for each sample: each library was sequenced on two lanes. Recall that in our workflow, we had merged these FASTQ files prior to mapping, but here we are using the table based on the full dataset produced by Matthew.

So, we will go ahead and merge these replicates now, by simply summing the counts:

counts.t <- t(counts)
rownames(counts.t) <- names(raw_counts)[7:36]
sums <- rowsum(counts.t, group = rownames(counts.t))
counts <- t(sums)

(Alternatively, one could use the specialized function DESeq2::collapseReplicates() for this.)

Check sample IDs

For differential expression analysis, we will be using the popularDESeq2 R/Bioconductor package (paper, website).

We will load both the count table and the metadata into DESeq2. When doing so, DESeq2 assumes that sample IDs in both tables match and are provided in the same order. Let’s make sure this is indeed the case.

Sort both data frames alphabetically:

metadata <- metadata[order(metadata$SampleID), ]
counts <- counts[, order(colnames(counts))]

Check if names are the same:

metadata$SampleID
##  [1] "C6_myb2" "C6_myb3" "C6_myb4" "cnt_1"   "cnt_2"   "cnt_3"   "CT_05"  
##  [8] "CT_06"   "CT_09"   "myb_1"   "myb_2"   "myb_3"   "T_26"    "T_29"   
## [15] "T_30"
colnames(counts)
##  [1] "C6_myb2" "C6_myb3" "C6_myb4" "cnt_1"   "cnt_2"   "cnt_3"   "CT_05"  
##  [8] "CT_06"   "CT_09"   "myb_1"   "myb_2"   "myb_3"   "T_26"    "T_29"   
## [15] "T_30"
matching_names <- identical(metadata$SampleID, colnames(counts))
matching_names
## [1] TRUE
if(matching_names == FALSE) stop("Sample ID in metadata and count matrix do not match!")

Create the DESeq2 object

We will create the DESeq2 object using the function DESeqDataSetFromMatrix(), which we will provide with three pieces of information:

  • The count data with argument countData.
  • The metadata with argument colData.
  • The model design for the DE analysis – argument design.
    For now, we will specify ~1, which means “no design” – we will change this before the actual DE analysis.
dds_raw <- DESeqDataSetFromMatrix(countData = counts,
                                  colData = metadata,
                                  design = ~ 1)

Remove the Ri_noexp group

#dds_raw <- dds_raw[, -which(dds_raw@colData$group == "Ri_noexp")]
#dds_raw@colData$group <- droplevels(dds_raw@colData$group)

dds_raw <- dds_raw[, -which(dds_raw@colData$Treatment == "noexp")]
#dds_raw@colData$group <- droplevels(dds_raw@colData$group)



Explore the count data

What are number of rows and columns of the count matrix?

dim(counts)
## [1] 34688    15

How many genes have non-zero counts?

dim(counts[rowSums(counts) > 0, ])
## [1] 28145    15

How many genes have total counts of at least 10?

dim(counts[rowSums(counts) >= 10, ])
## [1] 24771    15

Histogram of gene counts

Let’s plot a histogram of gene counts:

theme_set(theme_bw())

summed_gene_counts <- data.frame(gene_count = rowSums(counts)) %>%
  rownames_to_column("gene_id")

ggplot(data = summed_gene_counts) +
  geom_histogram(aes(x = gene_count), binwidth = 10000) +
  scale_y_log10(expand = c(0, 0)) +
  scale_x_continuous(expand = c(0,0))

Zoom in a bit:

ggplot(data = summed_gene_counts) +
  geom_histogram(aes(x = gene_count), binwidth = 1000) +
  scale_y_log10(expand = c(0, 0)) +
  scale_x_continuous(limits = c(0, 200000), expand = c(0,0)) +
  theme(plot.margin = margin(0.5, 0.7, 0.5, 0.5, "cm"))

How are counts distributed across samples? That is, we would like a sum of counts for each column. To get this, we use the apply() function, which can apply a function (in our case sum()) to all columns (hence MARGIN = 2 – for rows, use 1) of our counts dataframe:

apply(X = counts, MARGIN = 2, FUN = sum)
##  C6_myb2  C6_myb3  C6_myb4    cnt_1    cnt_2    cnt_3    CT_05    CT_06 
## 18122406 21913320 13659702 18844494 18890045 16634169 22183005 20620412 
##    CT_09    myb_1    myb_2    myb_3     T_26     T_29     T_30 
## 22924853 22070790 22829207 23195471 19902716 23221528 15793928



Principal Component Analysis (PCA)

Run the PCA and prepare for plotting

First, we normalize the count data to have even sampling across samples (with respect to library size) and approximately even variance:

vsd <- varianceStabilizingTransformation(dds_raw, blind = TRUE)

Next, we run the PCA and retrieve the data to plot with ggplot2:

pcaData <- plotPCA(vsd,
                   ntop = 500,
                   intgroup = c("AMF", "Treatment"),
                   returnData = TRUE)

We extract the percentage of variance explained by different principal components, so we can later add this information to the plot:

percentVar <- round(100 * attr(pcaData, "percentVar"))
percentVar
## [1] 58 16

We create a plot title with the species name in italic using the somewhat bizarre expression() function:

plot_title <- expression("PCA of " * italic(Glycine ~ max) * " transcript count data")

Plot the PCA results

ggplot(pcaData,
       aes(x = PC1, y = PC2, color = AMF, shape = Treatment)) +
  geom_point(size = 6) +
  xlab(paste0("PC1: ", percentVar[1], "% of variance")) +
  ylab(paste0("PC2: ", percentVar[2], "% of variance")) +
  ggtitle(plot_title)

Plot again – with sample names

library(ggrepel)

pca_plot2 <- ggplot(pcaData,
              aes(PC1, PC2, color = AMF, shape = Treatment)) +
  geom_point(size = 3) +
  geom_label_repel(aes(label = name)) +
  xlab(paste0("PC1: ", percentVar[1], "% of variance")) +
  ylab(paste0("PC2: ", percentVar[2], "% of variance")) +
  ggtitle(plot_title)

print(pca_plot2)



DE analysis – full dataset

The design has two factors: AMF and Treatment. Rather than fit a multivariate model, we can start by merging the two into a single factor called group, and fit a univariate model with this factor.

dds_raw$group <- factor(paste(dds_raw$AMF, dds_raw$Treatment, sep = "_"))
table(dds_raw$group)
## 
## control_mock  control_myb      Ri_mock       Ri_myb 
##            3            3            3            3

We will set the “reference” level of the factor to be the double negative control (empty substrate, no Agrobacteria):

dds_raw$group <- relevel(dds_raw$group, ref = "control_mock")
dds_raw$group
##  [1] control_mock control_mock control_mock Ri_mock      Ri_mock     
##  [6] Ri_mock      control_myb  control_myb  control_myb  Ri_myb      
## [11] Ri_myb       Ri_myb      
## Levels: control_mock control_myb Ri_mock Ri_myb

Next, we set the analysis design:

design(dds_raw) <- formula(~ group)

And finally, we perform the differential expression analysis with the DEseq() function:

dds <- DESeq(dds_raw)

The DESeq() function above performs three steps consecutively:

  • estimateSizeFactors() – “Normalization” by library size and composition.

    Note that DESeq2 doesn’t actually normalize the counts in the sense that it produces a matrix with adjusted counts. Instead it uses raw counts and includes the size factors in the modeling.

    To learn more about gene count normalization, see this video and this page.

  • estimateDispersions() – Estimate gene-wise dispersion (variance in counts).

  • nbinomWaldTest(ddsObj) – Fit the negative binomial GLM and calculate Wald statistics, which is the test statistic underlying the p-value for whether a gene is differentially expressed.

These functions could also be called separately, which would be useful if you want to be able to change more defaults.

The results table

res <- results(dds)
kable(head(res))
baseMean log2FoldChange lfcSE stat pvalue padj
Solyc00g160260.1 0.00000 NA NA NA NA NA
Solyc00g160270.1 0.00000 NA NA NA NA NA
Solyc00g500003.1 11.11875 -0.3323131 1.218149 -0.2728018 0.7850056 0.8764988
Solyc00g500004.1 0.00000 NA NA NA NA NA
Solyc00g500005.1 0.00000 NA NA NA NA NA
Solyc00g500006.1 0.00000 NA NA NA NA NA

By default, the results table prints statistics comparing the last level of the factor with the first level: that is, log-fold change and p-values describe differences between these two levels specifically. However, we can easily extract equivalent statistics for any pairwise comparison among our factor levels, which we will see later.

For now, we will explore what each column in this table means:

  • The baseMean column contains the mean expression level across all samples.

  • The log2FoldChange column contains the log2-fold change of gene counts between the compared levels, that is, it represents the effect size.

    A log2-fold change of 1 indicates that the expression in the reference level is two-fold lower than that of the other level, a log2-fold change of 2 indicates a four-fold difference, a log2-fold change of 3 indicates an eight-fold difference, and so on.

    Similarly, negative log2-fold values indicate a change in gene counts in the other direction: the reference level is higher than the other level.

  • The lfcSE column indicates the uncertainty in terms of the standard error (SE) of the log2-fold change estimate.

  • The stat column indicates the value for the Wald test’s test statistic.

  • The pvalue column reported the uncorrected p-value from the Wald test.

  • Because we are testing significance for many genes, we need to correct for multiple testing. DESeq2 uses the Benjamini-Hochberg False Discovery Rate (FDR) correction, and these values are reported in the column padj (i.e., adjusted p-value).

A summary of this information about each column can be seen by running the mcols() function:

mcols(res)
## DataFrame with 6 rows and 2 columns
##                        type            description
##                 <character>            <character>
## baseMean       intermediate mean of normalized c..
## log2FoldChange      results log2 fold change (ML..
## lfcSE               results standard error: grou..
## stat                results Wald statistic: grou..
## pvalue              results Wald test p-value: g..
## padj                results   BH adjusted p-values

NA values in the results table

Some values in the results table can be set to NA for one of the following reasons:

  • If a gene contains a sample with a count outlier, both the p-value and adjusted p-value will be set to NA. (DESeq2 performs outlier detection using Cook’s distance.)

  • If all samples have zero counts for a given gene, the baseMean column will be zero, and the log2-fold change estimates, p-value and adjusted p-value will all be set to NA.

  • DESeq2 also automatically filters genes with a low mean count in the sense that it does not include them in the multiple testing correction. Therefore, in such cases, the p-value will not be NA, but the adjusted p-value will be.

    Because we have very low power to detect differential expression for such low-count genes, it is beneficial to remove them prior to the multiple testing correction: that way, the correction becomes less severe for the remaining genes.

Let’s see how many genes have NA p-values:

# Number of genes with NA p-value:
sum(is.na(res$pvalue))
## [1] 6990
# As a proportion of the total number of genes in the test:
sum(is.na(res$pvalue)) / nrow(res)
## [1] 0.2015106

And NA adjusted p-values:

# Number of genes with NA p-value:
sum(is.na(res$padj))
## [1] 10709
# As a proportion of the total number of genes in the test:
sum(is.na(res$padj)) / nrow(res)
## [1] 0.3087235



DE analysis – contrast two custom groups

Using the resultsNames function, we can see which pairwise contrasts between different levels of the factor are available (though it is not displayed in a particularly readable fashion):

resultsNames(dds)
## [1] "Intercept"                         "group_control_myb_vs_control_mock"
## [3] "group_Ri_mock_vs_control_mock"     "group_Ri_myb_vs_control_mock"

Not all pairwise contrasts between the 5 levels in our group factor are available here: instead, control_mock, which we set as the reference level, is being compared with the other 3 levels. (However, we can make other pairwise comparisons, too.)

Above, we looked at the results for the last of these comparisons (group_Ri_myb_vs_control_mock, i.e. “Ri_myb” vs. “control_mock”), simply because DESeq2 will show the last comparison by default when calling the results() function.

To see the results table for one of the other 3 comparisons, we pass a vector to the contrast argument of the results() function with the factor (group) and the two levels to be contrasted. For example, to see the results for “Ri_mock” vs. “control_mock”:

# Here, we could specify *any* pairwise contrast,
# not just the ones with "control_mock" that resultsNames() prints as seen above.
my_contrast <- c("Ri_mock", "control_mock")

res <- results(dds,
               contrast = c("group", my_contrast))

How many adjusted p-values were less than 0.1?

sum(res$padj < 0.1, na.rm = TRUE)
## [1] 5217

We’ll also create an object with adjusted (shrunken) LFC estimates, which will be useful for visualization and ranking of genes by LFC:

my_coef <- paste0("group_", paste0(my_contrast, collapse = "_vs_"))
my_coef
## [1] "group_Ri_mock_vs_control_mock"
res_LFC <- lfcShrink(dds,
                     coef = my_coef,
                     type = "apeglm",
                     lfcThreshold = 1)

Here, we had to provide the contrast (“coefficient”) in the format given by resultsNames(dds). (And that looks a bit confusing because this format uses underscores to separate levels, while our factor levels themselves also contain underscores.)

We also specified a threshold of 1 for the LFC value (lfcThreshold = 1), so we get s-values (analogous to p-values) that test not for differential expression of any magnitude (as in the tests above), but whether the LFC is greater than our specified threshold:

head(res_LFC)
## log2 fold change (MAP): group Ri mock vs control mock 
##  
## DataFrame with 6 rows and 4 columns
##                   baseMean log2FoldChange     lfcSE    svalue
##                  <numeric>      <numeric> <numeric> <numeric>
## Solyc00g160260.1    0.0000             NA        NA        NA
## Solyc00g160270.1    0.0000             NA        NA        NA
## Solyc00g500003.1   11.1187      -0.250441  0.536045  0.576886
## Solyc00g500004.1    0.0000             NA        NA        NA
## Solyc00g500005.1    0.0000             NA        NA        NA
## Solyc00g500006.1    0.0000             NA        NA        NA
sum(res_LFC$svalue < 0.1, na.rm = TRUE)
## [1] 1441

This can be a useful way to try and tease out statistical from biological significance.



Visually exploring the results

We will create a few plots, by way of example, of the results for the “Ri_mock” versus “control_mock” comparison, which we extracted above.

MA-plot:

For a nice overview of the results, we can plot a so-called “MA plot”. An MA plot shows, for each gene:

  • Count differences in terms of LFC between two groups, on the y-axis.

  • Mean counts across both groups, on the x-axis.

We can create an MA plot using DESeq2’s plotMA function, with significantly differentially expressed genes displayed in blue:

plotMA(res, ylim = c(-5, 5))

To be able to customize the plot, we’ll use returnData = TRUE like we have done with previous plots, and then plot the resulting dataframe with ggplot2:

d <- plotMA(res, returnData = TRUE)

ggplot(d, aes(x = mean, y = lfc, color = isDE)) +
  geom_point(size = 0.5) +
  scale_x_log10() +
  scale_y_continuous(limits = c(-10, 10)) +
  scale_color_manual(values = c('grey50', 'blue')) +
  guides(color = FALSE) +
  labs(x = "Mean of normalized counts",
       y = "LFC")

We can see that lowly-expressed genes tend to deviate from an LFC of 0 (same mean expression levels in the two groups) much more than highly-expressed genes do. However, this is an artifact of noise overpowering the signal when expression values are low. We can also see that no genes in the far left part of the plot are differentially expressed: this is due to this same lack of power.

DESeq2 provides several methods to adjust LFC estimates for this low-expression bias. We used one of those (lfcShrink()) above to produce the res_LFC object. Let’s create another MA plot with these adjusted LFC estimates:

d <- plotMA(res_LFC, ylim = c(-5, 5), returnData = TRUE)

ggplot(d, aes(x = mean, y = lfc, color = isDE)) +
  geom_point(size = 0.5) +
  scale_x_log10() +
  scale_color_manual(values = c('grey50', 'blue')) +
  guides(color = FALSE) +
  labs(x = "Mean of normalized counts",
       y = "Shrunken LFC")

Note that significance is now specified for LFC > 1 – FINISH

Finally, for a plot like this, it could be useful to be able to identify individual genes. There are way too many to print the gene names on the plot, though. Instead, we can also make the plot interactive with Plotly, so we can see the identity of each gene when we hover over the point:

library(plotly)

# To show the gene name, we need to have a column with gene names.
# Currently, the gene names are row names but ggplot2 (and other tidyverse
# applications) don't like that, so we create a column with gene names:
d$gene <- rownames(d)

# First we create a very similar ggplot to what we did above,
# but we assign gene names to "text":
p_ma <- ggplot(d,
               aes(x = mean, y = lfc, color = isDE, text = gene)) +
  geom_point(size = 0.5) +
  scale_x_log10() +
  scale_color_manual(values = c('grey50', 'blue')) +
  guides(color = FALSE) +
  labs(x = "Mean of normalized counts",
       y = "Shrunken LFC")

# Finally, we make the plot interactive and tell Plotly that it should show
# the "text" (i.e. gene names) as the "tooltip", meaning upon hovering:
ggplotly(p_ma, tooltip = "text")

Plot specific genes

We can also create plot of expression levels for individual genes. That is especially interesting for genes with highly significant differential expression.

Let’s plot the top-5 most significantly differentially expressed genes:

# First, we select the 5 genes with the lowest adjusted p-value:
top5 <- row.names(res[order(res$padj)[1:5], ])
# Then we create a function to make a plot for a single gene:
plotgene <- function(geneID, dds) {
  
  d <- plotCounts(dds,
                  gene = geneID,
                  intgroup = "group",
                  returnData = TRUE)

  p <- ggplot(d, aes(x = group, y = count)) +
          geom_point(position = position_jitter(w = 0.1, h = 0)) +
          labs(title = geneID) +
          theme_bw()
  
  print(p)
}

Finally, we use sapply() to apply this function to each of our genes in the top5 vector.

none <- sapply(top5, plotgene, dds)

If we wanted to, we could easily create plots for 100s of genes, this way.

Heatmap

We can create heatmaps with the pheatmap function. Let’s start by creating a function that will plot a heatmap given a vector of gene IDs and a DESeq2 object dds:

plot_heatmap <- function(geneIDs, dds) {
  
  ntd <- assay(normTransform(dds))
  
  ntd_sel <- ntd[match(geneIDs, rownames(ntd)), ]
  df_meta <- as.data.frame(colData(dds)[, c("AMF", "Treatment")])
  
  pheatmap(ntd_sel,
         cluster_rows = FALSE,
         cluster_cols = FALSE,
         show_rownames = FALSE,
         annotation_col = df_meta)
}

Now, we can easily create a heatmap for the top-20 most highly differentially expressed genes:

top20_DE <- row.names(res[order(res$padj)[1:20], ])
plot_heatmap(top20_DE, dds)

Or for the 20 most highly expressed genes:

top20_hi_idx <- order(rowMeans(counts(dds, normalized = TRUE)),
                  decreasing = TRUE)[1:20]
top20_hi <- row.names(dds)[top20_hi_idx]
plot_heatmap(top20_hi, dds)

Export the results

Let’s save the results dataframe to file.

Note that it will only contain the results for one comparison. Also, if we write the results dataframe to file, we won’t be able to tell from the file what the comparison is, so let’s store that in two columns:

my_contrast
## [1] "Ri_mock"      "control_mock"
my_contrast_pasted <- paste0(my_contrast, collapse = "_vs_")
my_contrast_pasted
## [1] "Ri_mock_vs_control_mock"
res$level1 <- my_contrast[1]
res$level2 <- my_contrast[2]
kable(head(res))
baseMean log2FoldChange lfcSE stat pvalue padj level1 level2
Solyc00g160260.1 0.00000 NA NA NA NA NA Ri_mock control_mock
Solyc00g160270.1 0.00000 NA NA NA NA NA Ri_mock control_mock
Solyc00g500003.1 11.11875 -1.917985 1.255117 -1.528132 0.1264797 0.3040457 Ri_mock control_mock
Solyc00g500004.1 0.00000 NA NA NA NA NA Ri_mock control_mock
Solyc00g500005.1 0.00000 NA NA NA NA NA Ri_mock control_mock
Solyc00g500006.1 0.00000 NA NA NA NA NA Ri_mock control_mock

Now we can write res to file:

res_file <- file.path(outdir, paste0(my_contrast_pasted, '_all-res.txt'))

write.table(res, res_file,
            sep = '\t', row.names = TRUE, quote = FALSE)

We may also want to save a separate table with only significant results:

res_sig_file <- file.path(outdir, paste0(my_contrast_pasted, '_sig-res.txt'))

res_sig <- res %>%
  as.data.frame() %>% 
  dplyr::filter(padj < 0.1)

write.table(res_sig, res_sig_file,
            sep = '\t', row.names = TRUE, quote = FALSE)



DE analysis – all pairwise comparisons

To run the DE analysis for all pairwise comparisons, we will start by writing a function that takes a comparison (contrast) in the form of c(level1, level2) along with a DESeq2 object (dds), and output a data frame with significantly differentially expressed genes:

sig_contrast <- function(my_contrast, dds) {
  
  res_sig <- results(dds,
                     contrast = c("group", my_contrast)) %>%
    as.data.frame() %>% 
    dplyr::filter(padj < 0.1) %>%
    mutate(level1 = my_contrast[1],
           level2 = my_contrast[2])
  
  cat(my_contrast[1], "versus", my_contrast[2], ":", nrow(res_sig), "significant\n")
  
  return(res_sig)
}

Then, we will create a list with all pairwise combinations of our group factor using the combn() function:

comps <- combn(levels(dds@colData$group), 2, simplify = FALSE)

Finally, we run the function for all of our pairwise comparisons, and using do.call(rbind, ...), we concatenate all the results in a single data frame:

sig_all_contrasts <- do.call(rbind, lapply(comps, sig_contrast, dds))
## control_mock versus control_myb : 8327 significant
## control_mock versus Ri_mock : 5217 significant
## control_mock versus Ri_myb : 6109 significant
## control_myb versus Ri_mock : 4223 significant
## control_myb versus Ri_myb : 2263 significant
## Ri_mock versus Ri_myb : 388 significant
# The `kable()` function is just to display tables nicely in this R Markdown
# document -- use `head(sig_all_contrasts)` in your own code.
kable(head(sig_all_contrasts))
baseMean log2FoldChange lfcSE stat pvalue padj level1 level2
61.886942 2.682127 0.5736071 4.675896 0.0000029 0.0001509 control_mock control_myb
3.043385 5.914974 2.2213307 2.662806 0.0077492 0.0347193 control_mock control_myb
2.158579 5.117064 1.8784057 2.724153 0.0064467 0.0305506 control_mock control_myb
4393.208985 1.725896 0.3936111 4.384775 0.0000116 0.0003991 control_mock control_myb
82.440348 1.095528 0.3450265 3.175200 0.0014973 0.0109290 control_mock control_myb
8.675927 1.984984 0.7470738 2.657012 0.0078837 0.0351581 control_mock control_myb

DE analysis – with two factors

Controlling for one factor

Say we wanted to analyze the effect of “mock” versus “myb” (“Treatment” column) while controlling for the effects of “control” versus “Ri” (“AMF” column).

Let’s start by turning “Treatment” and “AMF” into factors, and saving a new DESeq2 object:

# Convert Treatment and AMF into factors: 
dds_raw$Treatment <- relevel(factor(dds_raw$Treatment), ref = "mock")
dds_raw$AMF <- relevel(factor(dds_raw$AMF), ref = "control")

# Save a new object:
dds_2f_raw <- dds_raw

To include both factors, we use a + in the formula. Note that the order matters: using AMF + Treatment, we test for the effect of “Treatment” (the last factor), while controlling for the effect of AMF (the first factor).

design(dds_2f_raw) <- formula(~ AMF + Treatment)

Run DESeq with the new design:

dds_2f <- DESeq(dds_2f_raw)
res <- results(dds_2f)

How many adjusted p-values were less than 0.1?

sum(res$padj < 0.1, na.rm = TRUE)
## [1] 3719

With an interaction term

We can add an interaction term using notation like level1:level2, with a colon. In this case, we want to include both “AMF”, “Treatment”, and the interaction between the two:

# Save a new object:
dds_2fi_raw <- dds_raw

# The interaction term is `AMF:Treatment`:
design(dds_2fi_raw) <- formula(~ AMF + Treatment + AMF:Treatment)

Run DESeq with the new design:

dds_2fi <- DESeq(dds_2fi_raw)

resultsNames(dds_2fi)
## [1] "Intercept"             "AMF_Ri_vs_control"     "Treatment_myb_vs_mock"
## [4] "AMFRi.Treatmentmyb"

From the DESeq2 vignette: >The key point to remember about designs with interaction terms is that, unlike for a design ~ genotype + condition, where the condition effect represents the overall effect controlling for differences due to genotype, by adding genotype:condition, the main condition effect only represents the effect of condition for the reference level of genotype (I, or whichever level was defined by the user as the reference level). The interaction terms genotypeII.conditionB and genotypeIII.conditionB give the difference between the condition effect for a given genotype and the condition effect for the reference genotype.

Now, we can check a number of different effects:

Does the effects of Treatment differ among levels of AMF?

This is described by the interaction term, which came last in our formula. Therefore, the results() function will get results for the interaction term by default:

res <- results(dds_2fi)
kable(head(res))
baseMean log2FoldChange lfcSE stat pvalue padj
Solyc00g160260.1 0.00000 NA NA NA NA NA
Solyc00g160270.1 0.00000 NA NA NA NA NA
Solyc00g500003.1 11.11875 1.971865 1.747953 1.128099 0.2592781 0.5103138
Solyc00g500004.1 0.00000 NA NA NA NA NA
Solyc00g500005.1 0.00000 NA NA NA NA NA
Solyc00g500006.1 0.00000 NA NA NA NA NA

The effect of Treatment in “control” AMF:

res <- results(dds_2fi,
               contrast = c("Treatment", "myb", "mock"))
kable(head(res))
baseMean log2FoldChange lfcSE stat pvalue padj
Solyc00g160260.1 0.00000 NA NA NA NA NA
Solyc00g160270.1 0.00000 NA NA NA NA NA
Solyc00g500003.1 11.11875 -0.3861962 1.215878 -0.3176274 0.7507676 0.8455709
Solyc00g500004.1 0.00000 NA NA NA NA NA
Solyc00g500005.1 0.00000 NA NA NA NA NA
Solyc00g500006.1 0.00000 NA NA NA NA NA

The effect of Treatment in “myb” AMF:

res <- results(dds_2fi,
               contrast = list(c("Treatment_myb_vs_mock", "AMFRi.Treatmentmyb")))
kable(head(res))
baseMean log2FoldChange lfcSE stat pvalue padj
Solyc00g160260.1 0.00000 NA NA NA NA NA
Solyc00g160270.1 0.00000 NA NA NA NA NA
Solyc00g500003.1 11.11875 1.585668 1.255784 1.262692 0.2066998 0.9374357
Solyc00g500004.1 0.00000 NA NA NA NA NA
Solyc00g500005.1 0.00000 NA NA NA NA NA
Solyc00g500006.1 0.00000 NA NA NA NA NA

The effect of AMF in the “mock” Treatment:

res <- results(dds_2fi,
                contrast = c("AMF", "Ri", "control"))
kable(head(res))
baseMean log2FoldChange lfcSE stat pvalue padj
Solyc00g160260.1 0.00000 NA NA NA NA NA
Solyc00g160270.1 0.00000 NA NA NA NA NA
Solyc00g500003.1 11.11875 -1.917982 1.255115 -1.528132 0.1264798 0.3040459
Solyc00g500004.1 0.00000 NA NA NA NA NA
Solyc00g500005.1 0.00000 NA NA NA NA NA
Solyc00g500006.1 0.00000 NA NA NA NA NA

The effect of AMF in “myb” Treatment:

res <- results(dds_2fi,
               contrast = list(c("AMF_Ri_vs_control", "AMFRi.Treatmentmyb")))
kable(head(res))
baseMean log2FoldChange lfcSE stat pvalue padj
Solyc00g160260.1 0.00000 NA NA NA NA NA
Solyc00g160270.1 0.00000 NA NA NA NA NA
Solyc00g500003.1 11.11875 0.0538826 1.216568 0.0442907 0.9646727 0.9859513
Solyc00g500004.1 0.00000 NA NA NA NA NA
Solyc00g500005.1 0.00000 NA NA NA NA NA
Solyc00g500006.1 0.00000 NA NA NA NA NA
LS0tCnRpdGxlOiAiRGlmZmVyZW50aWFsIGV4cHJlc3Npb24gKERFKSBhbmFseXNpcyA8YnI+IHdpdGggREVTZXEyIgphdXRob3I6ICJKZWxtZXIgUG9lbHN0cmEiCmRhdGU6ICIzLzI2LzIwMjEiCm91dHB1dDoKICBodG1sX2RvY3VtZW50OgogICAgY29kZV9kb3dubG9hZDogdHJ1ZQogICAgdG9jX2RlcHRoOiAzCiAgICB0b2M6IHRydWUKICAgIHRvY19mbG9hdDogdHJ1ZQogICAgdGhlbWU6IGNlcnVsZWFuCiAgICBoaWdobGlnaHQ6IHRhbmdvCiAgICBhbmNob3Jfc2VjdGlvbnM6IHRydWUKICAgIGRmX3ByaW50OiBrYWJsZQogICAgY3NzOiBodG1sX3BhZ2UuY3NzCmVkaXRvcl9vcHRpb25zOiAKICBjaHVua19vdXRwdXRfdHlwZTogaW5saW5lCi0tLQoKYGBge3Igc2V0dXAsIGluY2x1ZGU9RkFMU0V9CmtuaXRyOjpvcHRzX2NodW5rJHNldCgKICBlY2hvPVRSVUUsIGV2YWw9VFJVRSwgY2FjaGU9VFJVRSwKICB3YXJuaW5nPUZBTFNFLCBtZXNzYWdlPUZBTFNFLAogIGNsYXNzLnNvdXJjZSA9ICJyX2NvZGUiLAogIGNsYXNzLm91dHB1dCA9ICJyX291dHB1dCIsCiAgY2xhc3Mud2FybmluZyA9ICJyX3dhcm5pbmciLAogIGNsYXNzLm1lc3NhZ2UgPSAicl93YXJuaW5nIiwKICBjbGFzcy5lcnJvciA9ICJyX2Vycm9yIgogICkKYGBgCgo8YnI+IDxicj4KCi0tLS0KCiMjIEdldHRpbmcgc2V0IHVwCgojIyMgU3RhcnQgYW4gUlN0dWRpbyBzZXNzaW9uIGF0IE9TQwoKPGRldGFpbHM+PHN1bW1hcnk+U2hvdyBpbnN0cnVjdGlvbnM8L3N1bW1hcnk+CgotIExvZ2luIHRvIE9TQyBhdCA8aHR0cHM6Ly9vbmRlbWFuZC5vc2MuZWR1Pi4KCi0gQ2xpY2sgb24gYEludGVyYWN0aXZlIEFwcHNgICh0b3AgYmFyKSA+IGBSU3R1ZGlvIFNlcnZlciAoT3dlbnMgYW5kIFBpdHplcilgCgo8cCBhbGlnbj0iY2VudGVyIj4KPGltZyBzcmM9aW1nL2FwcHMucG5nIHdpZHRoPSI1MDAiPgo8L3A+CgotIEZpbGwgb3V0IHRoZSBmb3JtIGFzIHNob3duIFtoZXJlXShzbGlkZXMvMDMtT1NDLXNsaWRlcy5odG1sI3JzdHVkaW9fc2VydmVyX2pvYikuCgotIE5vdywgeW91IHNob3VsZCBzZWUgYSBib3ggbGlrZSB0aGlzOgoKPHAgYWxpZ249ImNlbnRlciI+CjxpbWcgc3JjPWltZy9vc2NfcXVldWVkLnBuZyB3aWR0aD0iNzAwIj4KPC9wPgoKLSBZb3VyIGpvYiBzaG91bGQgc3RhcnQgcnVubmluZyBwcmV0dHkgc29vbiwgYW5kIHdoZW4gaXQncyByZWFkeSB0aGUgYm94IHNob3VsZCBsb29rIGxpa2UgdGhpczogCgo8cCBhbGlnbj0iY2VudGVyIj4KPGltZyBzcmM9aW1nL29zY19ydW5uaW5nLnBuZyB3aWR0aD0iNzAwIj4KPC9wPgoKLSBDbGljayBgQ29ubmVjdCB0byBSU3R1ZGlvIFNlcnZlcmAgYXQgdGhlIGJvdHRvbSBvZiB0aGUgYm94LCBhbmQgYW4gUlN0dWRpbyBTZXJ2ZXIgaW5zdGFuY2Ugd2lsbCBvcGVuLiBZb3UncmUgcmVhZHkgdG8gZ28hCgo8L2RldGFpbHM+CgojIyMgQ3JlYXRlIGFuIFJTdHVkaW8gUHJvamVjdAoKPGRldGFpbHM+PHN1bW1hcnk+U2hvdyBpbnN0cnVjdGlvbnM8L3N1bW1hcnk+CgpgYGB7ciwgZXZhbD1GQUxTRX0KYmFzZWRpciA8LSAiL2ZzL3Byb2plY3QvUEFTMDQ3MS90ZWFjaC9taXNjLzIwMjEtMDJfcm5hc2VxLyIKCiMgR2V0IHlvdXIgdXNlciBuYW1lIChieSBydW5uaW5nIGEgc2hlbGwgY29tbWFuZCB2aWEgdGhlIGBzeXN0ZW0oKWAgZnVuY3Rpb246Cm1lIDwtIHN5c3RlbSgiZWNobyAkVVNFUiIsIGludGVybiA9IFRSVUUpCgojIEJ1aWxkIHRoZSBwYXRoIHRvIHRoZSB0YXJnZXQgZGlyOgpwcm9qX2RpciA8LSBmaWxlLnBhdGgoYmFzZWRpciwgbWUpCgojIENyZWF0ZSB0aGUgUHJvamVjdDoKbGlicmFyeSh1c2V0aGlzKQpjcmVhdGVfcHJvamVjdChwYXRoID0gcHJval9kaXIpCmBgYAoKTm93LCBSU3R1ZGlvIHdpbGwgcmVsb2FkIHdpdGggdGhlIG5ld2x5IGNyZWF0ZWQgUHJvamVjdCBvcGVuLgoKSWYgeW91IGdldCB0aGUgcG9wLXVwIGJlbG93LCBjbGljayBgRG9uJ3QgU2F2ZWAgKGRvIHRoaXMgd2hlbmV2ZXIgeW91IGdldCB0aGF0IHBvcC11cCk6Cgo8cCBhbGlnbj0iY2VudGVyIj4KPGltZyBzcmM9aW1nL3JkYXRhLXBvcHVwLnBuZyB3aWR0aD0iMzUwIj4KPC9wPgoKPC9kZXRhaWxzPgoKIyMjIENvcHkgdGhlIGNvdW50IGRhdGEgYW5kIHRoZSBtZXRhZGF0YSBmaWxlcwoKR28gdGhlIGBUZXJtaW5hbGAgdGFiIG5leHQgdG8gdGhlIFIgYENvbnNvbGVgIGluIHRoZSBib3R0b20gbGVmdCBvZiB0aGUgUlN0dWRpbwp3aW5kb3c6CgpgYGB7c2gsIGV2YWw9RkFMU0V9CmNkIC9mcy9wcm9qZWN0L1BBUzA0NzEvdGVhY2gvbWlzYy8yMDIxLTAyX3JuYXNlcS8kVVNFUgoKY3AgLi4vbWFzdGVyL2RhdGEvbWV0YS9tZXRhZGF0YS50eHQgZGF0YS9tZXRhCmNwIC4uL21hc3Rlci9yZXN1bHRzL2NvdW50L2dlbmVfY291bnRzX2FsbC50eHQgcmVzdWx0cy9jb3VudApgYGAKCiMjIyBMb2FkIHRoZSBuZWNlc3NhcnkgcGFja2FnZXMKCmBgYHtyLCB3YXJuaW5nPUZBTFNFLCBtZXNzYWdlPUZBTFNFfQppZighICJwYWNtYW4iICVpbiUgaW5zdGFsbGVkLnBhY2thZ2VzKCkpIGluc3RhbGwucGFja2FnZXMoInBhY21hbiIpCgpwYWNrYWdlcyA8LSBjKCJERVNlcTIiLCAgICAgICAgICAjIERpZmZlcmVudGlhbCBleHByZXNzaW9uIGFuYWx5c2lzCiAgICAgICAgICAgICAgInRpZHl2ZXJzZSIsICAgICAgICMgTWlzYy4gZGF0YSBtYW5pcHVsYXRpb24gYW5kIHBsb3R0aW5nCiAgICAgICAgICAgICAgImhlcmUiLCAgICAgICAgICAgICMgTWFuYWdpbmcgZmlsZSBwYXRocwogICAgICAgICAgICAgICJwaGVhdG1hcCIsICAgICAgICAjIEhlYXRtYXAgcGxvdAogICAgICAgICAgICAgICJhcGVnbG0iLCAgICAgICAgICAjIEZvciBMRkMgc2hyaW5rYWdlCiAgICAgICAgICAgICAgImtuaXRyIikgICAgICAgICAgICMgRm9yIHRhYmxlIHByaW50aW5nIGluIE1hcmtkb3duIGZpbGUKcGFjbWFuOjpwX2xvYWQoY2hhciA9IHBhY2thZ2VzKQoKdGhlbWVfc2V0KHRoZW1lX2J3KCkpICAjIFNldCBnZ3Bsb3QgdGhlbWUKYGBgCgojIyMgSW5wdXQgYW5kIG91dHB1dCBkaXJzIGFuZCBmaWxlcwoKYGBge3IsIGV2YWw9VFJVRSwgZWNobz1GQUxTRX0KY291bnRfdGFibGVfZmlsZSA8LSAicmVzdWx0cy9jb3VudC9nZW5lX2NvdW50c19hbGwudHh0IgptZXRhZGF0YV9maWxlIDwtICJkYXRhL21ldGEvbWV0YWRhdGEudHh0IgoKb3V0ZGlyIDwtICJyZXN1bHRzL0RFLyIKcGxvdGRpciA8LSAicmVzdWx0cy9ERS9maWcvIgpgYGAKCklucHV0IGZpbGVzOgoKLSAqKkdlbmUgY291bnRzIHRhYmxlKiogLS0gdGhlIGZpbGUgZXhhY3RseSBhcyBpdCB3YXMgd3JpdHRlbiBieSAqZmVhdHVyZUNvdW50cyouCiAgCi0gKipNZXRhZGF0YSB0YWJsZSoqIC0tIHNvIHdlIGNhbiBncm91cCBvdXIgc2FtcGxlcyBhbmQgbWFrZSBjb21wYXJpc29ucyBiZXR3ZWVuCiAgdGhlc2UgZ3JvdXBzLgoKYGBge3IsIGV2YWw9RkFMU0V9CmNvdW50X3RhYmxlX2ZpbGUgPC0gaGVyZSgicmVzdWx0cy9jb3VudC9nZW5lX2NvdW50c19hbGwudHh0IikKbWV0YWRhdGFfZmlsZSA8LSBoZXJlKCJkYXRhL21ldGEvbWV0YWRhdGEudHh0IikKYGBgCgpPdXRwdXQgZGlyZWN0b3JpZXMgLS0gYW5kIHdlIGNyZWF0ZSB0aGVtIGlmIHRoZXkgZG9uJ3QgYWxyZWFkeSBleGlzdDoKCmBgYHtyfQpvdXRkaXIgPC0gaGVyZSgicmVzdWx0cy9ERS8iKQpwbG90ZGlyIDwtIGhlcmUoInJlc3VsdHMvREUvZmlnLyIpCgppZiAoIWRpci5leGlzdHMocGxvdGRpcikpIGRpci5jcmVhdGUocGxvdGRpciwgcmVjdXJzaXZlID0gVFJVRSkKaWYgKCFkaXIuZXhpc3RzKHBsb3RkaXIpKSBkaXIuY3JlYXRlKHBsb3RkaXIsIHJlY3Vyc2l2ZSA9IFRSVUUpCmBgYAoKIyMjIExvYWQgaW5wdXQgZGF0YQoKTG9hZCB0aGUgY291bnQgdGFibGUgZnJvbSAqZmVhdHVyZUNvdW50cyo6CgpgYGB7ciwgZXZhbD1UUlVFfQpyYXdfY291bnRzIDwtIHJlYWQudGFibGUoY291bnRfdGFibGVfZmlsZSwKICAgICAgICAgICAgICAgICAgICAgICAgIHNlcCA9ICJcdCIsIGhlYWRlciA9IFRSVUUsIHNraXAgPSAxKQpgYGAKCkxvYWQgdGhlIG1ldGFkYXRhIGluZm9ybWF0aW9uOgoKYGBge3J9Cm1ldGFkYXRhIDwtIHJlYWQudGFibGUobWV0YWRhdGFfZmlsZSwgaGVhZGVyID0gVFJVRSkKCmthYmxlKGhlYWQobWV0YWRhdGEpKQpgYGAKClRoZSBgVHJlYXRtZW50YCBjb2x1bW4gY3VycmVudGx5IGhhcyB0aGUgdmFsdWVzIGBBZ3JvYmFjdGVyaXVtX25vZXhwYCwKYEFncm9iYWN0ZXJpdW1fbXliYCwgYW5kIGBtb2NrYC4KClRvIHNob3J0ZW4gdGhpcyBhIGJpdCwgd2UnbGwgZ2V0IHJpZCBvZiAiQWdyb2JhY3Rlcml1bV8iOgoKYGBge3J9Cm1ldGFkYXRhJFRyZWF0bWVudCA8LSBzdWIoIkFncm9iYWN0ZXJpdW1fIiwgIiIsIG1ldGFkYXRhJFRyZWF0bWVudCkKCnVuaXF1ZShtZXRhZGF0YSRUcmVhdG1lbnQpCmBgYAoKCjxicj4KCi0tLS0KCiMjIFByZXBhcmUgdGhlIGRhdGEKCkNoYW5nZSB0aGUgY29sdW1uIG5hbWVzLCB3aGljaCBhcmUgdmVyeSBsb25nOgoKYGBge3J9CmNvbG5hbWVzKHJhd19jb3VudHMpWzc6OF0KYGBgCgpgYGB7cn0KbXlfcmVnZXggPC0gIi4rUG9uY2VNXyguKylfVjFOLisiCmNvbG5hbWVzKHJhd19jb3VudHMpIDwtIHN1YihteV9yZWdleCwgIlxcMSIsIGNvbG5hbWVzKHJhd19jb3VudHMpKQoKY29sbmFtZXMocmF3X2NvdW50cykKYGBgCgpCZXNpZGVzIHRoZSBjb3VudHMsIHRoZXJlIGFyZSBjb2x1bW5zIHdpdGggbWV0YWRhdGEgZm9yIGVhY2ggZ2VuZToKCmBgYHtyfQpyYXdfY291bnRzWzE6NSwgMTo4XQpgYGAKCkxldCdzIHJlbW92ZSB0aG9zZToKCmBgYHtyfQpjb3VudHMgPC0gcmF3X2NvdW50c1ssIDc6bmNvbChyYXdfY291bnRzKV0Kcm93bmFtZXMoY291bnRzKSA8LSByYXdfY291bnRzJEdlbmVpZApgYGAKCiMjIyBSZXBsaWNhdGUgc2FtcGxlcwoKSW4gdGhpcyB0YWJsZSwgdGhlcmUgYXJlIHR3byBzZXBhcmF0ZSBlbnRyaWVzIGZvciBlYWNoIHNhbXBsZToKZWFjaCBsaWJyYXJ5IHdhcyBzZXF1ZW5jZWQgb24gdHdvIGxhbmVzLgpSZWNhbGwgdGhhdCBpbiBvdXIgd29ya2Zsb3csIHdlIGhhZCBtZXJnZWQgdGhlc2UgRkFTVFEgZmlsZXMgcHJpb3IgdG8gbWFwcGluZywKYnV0IGhlcmUgd2UgYXJlIHVzaW5nIHRoZSB0YWJsZSBiYXNlZCBvbiB0aGUgZnVsbCBkYXRhc2V0IHByb2R1Y2VkIGJ5IE1hdHRoZXcuCgpTbywgd2Ugd2lsbCBnbyBhaGVhZCBhbmQgbWVyZ2UgdGhlc2UgcmVwbGljYXRlcyBub3csIGJ5IHNpbXBseSBzdW1taW5nIHRoZSBjb3VudHM6CgpgYGB7cn0KY291bnRzLnQgPC0gdChjb3VudHMpCnJvd25hbWVzKGNvdW50cy50KSA8LSBuYW1lcyhyYXdfY291bnRzKVs3OjM2XQpzdW1zIDwtIHJvd3N1bShjb3VudHMudCwgZ3JvdXAgPSByb3duYW1lcyhjb3VudHMudCkpCmNvdW50cyA8LSB0KHN1bXMpCmBgYAoKKEFsdGVybmF0aXZlbHksIG9uZSBjb3VsZCB1c2UgdGhlIHNwZWNpYWxpemVkIGZ1bmN0aW9uCmBERVNlcTI6OmNvbGxhcHNlUmVwbGljYXRlcygpYCBmb3IgdGhpcy4pCgojIyMgQ2hlY2sgc2FtcGxlIElEcwoKRm9yIGRpZmZlcmVudGlhbCBleHByZXNzaW9uIGFuYWx5c2lzLCB3ZSB3aWxsIGJlIHVzaW5nIHRoZSBwb3B1bGFyKipERVNlcTIqKgpSL0Jpb2NvbmR1Y3RvciBwYWNrYWdlCihbcGFwZXJdKGh0dHBzOi8vZ2Vub21lYmlvbG9neS5iaW9tZWRjZW50cmFsLmNvbS9hcnRpY2xlcy8xMC4xMTg2L3MxMzA1OS0wMTQtMDU1MC04KSwKW3dlYnNpdGVdKGh0dHBzOi8vYmlvY29uZHVjdG9yLm9yZy9wYWNrYWdlcy9yZWxlYXNlL2Jpb2MvaHRtbC9ERVNlcTIuaHRtbCkpLgoKV2Ugd2lsbCBsb2FkIGJvdGggdGhlIGNvdW50IHRhYmxlIGFuZCB0aGUgbWV0YWRhdGEgaW50byAqREVTZXEyKi4KV2hlbiBkb2luZyBzbywgKkRFU2VxMiogYXNzdW1lcyB0aGF0IHNhbXBsZSBJRHMgaW4gYm90aCB0YWJsZXMgbWF0Y2ggYW5kIAphcmUgcHJvdmlkZWQgaW4gdGhlIHNhbWUgb3JkZXIuIExldCdzIG1ha2Ugc3VyZSB0aGlzIGlzIGluZGVlZCB0aGUgY2FzZS4KClNvcnQgYm90aCBkYXRhIGZyYW1lcyBhbHBoYWJldGljYWxseToKCmBgYHtyfQptZXRhZGF0YSA8LSBtZXRhZGF0YVtvcmRlcihtZXRhZGF0YSRTYW1wbGVJRCksIF0KY291bnRzIDwtIGNvdW50c1ssIG9yZGVyKGNvbG5hbWVzKGNvdW50cykpXQpgYGAKCkNoZWNrIGlmIG5hbWVzIGFyZSB0aGUgc2FtZToKCmBgYHtyfQptZXRhZGF0YSRTYW1wbGVJRAoKY29sbmFtZXMoY291bnRzKQoKbWF0Y2hpbmdfbmFtZXMgPC0gaWRlbnRpY2FsKG1ldGFkYXRhJFNhbXBsZUlELCBjb2xuYW1lcyhjb3VudHMpKQptYXRjaGluZ19uYW1lcwppZihtYXRjaGluZ19uYW1lcyA9PSBGQUxTRSkgc3RvcCgiU2FtcGxlIElEIGluIG1ldGFkYXRhIGFuZCBjb3VudCBtYXRyaXggZG8gbm90IG1hdGNoISIpCmBgYAoKIyMjIENyZWF0ZSB0aGUgREVTZXEyIG9iamVjdAoKV2Ugd2lsbCBjcmVhdGUgdGhlIERFU2VxMiBvYmplY3QgdXNpbmcgdGhlIGZ1bmN0aW9uIGBERVNlcURhdGFTZXRGcm9tTWF0cml4KClgLAp3aGljaCB3ZSB3aWxsIHByb3ZpZGUgd2l0aCB0aHJlZSBwaWVjZXMgb2YgaW5mb3JtYXRpb246CgotIFRoZSBjb3VudCBkYXRhIHdpdGggYXJndW1lbnQgYGNvdW50RGF0YWAuCi0gVGhlIG1ldGFkYXRhIHdpdGggYXJndW1lbnQgYGNvbERhdGFgLgotIFRoZSBtb2RlbCBkZXNpZ24gZm9yIHRoZSBERSBhbmFseXNpcyAtLSBhcmd1bWVudCBgZGVzaWduYC4gIAogIEZvciBub3csIHdlIHdpbGwgc3BlY2lmeSBgfjFgLCB3aGljaCBtZWFucyAibm8gZGVzaWduIiAtLQogIHdlIHdpbGwgY2hhbmdlIHRoaXMgYmVmb3JlIHRoZSBhY3R1YWwgREUgYW5hbHlzaXMuCgpgYGB7cn0KZGRzX3JhdyA8LSBERVNlcURhdGFTZXRGcm9tTWF0cml4KGNvdW50RGF0YSA9IGNvdW50cywKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGNvbERhdGEgPSBtZXRhZGF0YSwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGRlc2lnbiA9IH4gMSkKYGBgCgojIyMgUmVtb3ZlIHRoZSBgUmlfbm9leHBgIGdyb3VwCgpgYGB7cn0KI2Rkc19yYXcgPC0gZGRzX3Jhd1ssIC13aGljaChkZHNfcmF3QGNvbERhdGEkZ3JvdXAgPT0gIlJpX25vZXhwIildCiNkZHNfcmF3QGNvbERhdGEkZ3JvdXAgPC0gZHJvcGxldmVscyhkZHNfcmF3QGNvbERhdGEkZ3JvdXApCgpkZHNfcmF3IDwtIGRkc19yYXdbLCAtd2hpY2goZGRzX3Jhd0Bjb2xEYXRhJFRyZWF0bWVudCA9PSAibm9leHAiKV0KI2Rkc19yYXdAY29sRGF0YSRncm91cCA8LSBkcm9wbGV2ZWxzKGRkc19yYXdAY29sRGF0YSRncm91cCkKYGBgCgo8YnI+CgotLS0tCgojIyBFeHBsb3JlIHRoZSBjb3VudCBkYXRhCgpXaGF0IGFyZSBudW1iZXIgb2Ygcm93cyBhbmQgY29sdW1ucyBvZiB0aGUgY291bnQgbWF0cml4PwoKYGBge3J9CmRpbShjb3VudHMpCmBgYAoKSG93IG1hbnkgZ2VuZXMgaGF2ZSBub24temVybyBjb3VudHM/CgpgYGB7cn0KZGltKGNvdW50c1tyb3dTdW1zKGNvdW50cykgPiAwLCBdKQoKYGBgCgpIb3cgbWFueSBnZW5lcyBoYXZlIHRvdGFsIGNvdW50cyBvZiBhdCBsZWFzdCAxMD8KCmBgYHtyfQpkaW0oY291bnRzW3Jvd1N1bXMoY291bnRzKSA+PSAxMCwgXSkKYGBgCgojIyMgSGlzdG9ncmFtIG9mIGdlbmUgY291bnRzCgpMZXQncyBwbG90IGEgaGlzdG9ncmFtIG9mIGdlbmUgY291bnRzOgoKYGBge3J9CnRoZW1lX3NldCh0aGVtZV9idygpKQoKc3VtbWVkX2dlbmVfY291bnRzIDwtIGRhdGEuZnJhbWUoZ2VuZV9jb3VudCA9IHJvd1N1bXMoY291bnRzKSkgJT4lCiAgcm93bmFtZXNfdG9fY29sdW1uKCJnZW5lX2lkIikKCmdncGxvdChkYXRhID0gc3VtbWVkX2dlbmVfY291bnRzKSArCiAgZ2VvbV9oaXN0b2dyYW0oYWVzKHggPSBnZW5lX2NvdW50KSwgYmlud2lkdGggPSAxMDAwMCkgKwogIHNjYWxlX3lfbG9nMTAoZXhwYW5kID0gYygwLCAwKSkgKwogIHNjYWxlX3hfY29udGludW91cyhleHBhbmQgPSBjKDAsMCkpCmBgYAoKWm9vbSBpbiBhIGJpdDoKCmBgYHtyfQpnZ3Bsb3QoZGF0YSA9IHN1bW1lZF9nZW5lX2NvdW50cykgKwogIGdlb21faGlzdG9ncmFtKGFlcyh4ID0gZ2VuZV9jb3VudCksIGJpbndpZHRoID0gMTAwMCkgKwogIHNjYWxlX3lfbG9nMTAoZXhwYW5kID0gYygwLCAwKSkgKwogIHNjYWxlX3hfY29udGludW91cyhsaW1pdHMgPSBjKDAsIDIwMDAwMCksIGV4cGFuZCA9IGMoMCwwKSkgKwogIHRoZW1lKHBsb3QubWFyZ2luID0gbWFyZ2luKDAuNSwgMC43LCAwLjUsIDAuNSwgImNtIikpCmBgYAoKSG93IGFyZSBjb3VudHMgZGlzdHJpYnV0ZWQgYWNyb3NzIHNhbXBsZXM/ClRoYXQgaXMsIHdlIHdvdWxkIGxpa2UgYSBzdW0gb2YgY291bnRzIGZvciBlYWNoIGNvbHVtbi4KVG8gZ2V0IHRoaXMsIHdlIHVzZSB0aGUgYGFwcGx5KClgIGZ1bmN0aW9uLCB3aGljaCBjYW4gYXBwbHkgYSBmdW5jdGlvbgooaW4gb3VyIGNhc2UgYHN1bSgpYCkgdG8gYWxsIGNvbHVtbnMgKGhlbmNlIGBNQVJHSU4gPSAyYCAtLSBmb3Igcm93cywgdXNlIGAxYCkKb2Ygb3VyIGBjb3VudHNgIGRhdGFmcmFtZToKCmBgYHtyfQphcHBseShYID0gY291bnRzLCBNQVJHSU4gPSAyLCBGVU4gPSBzdW0pCmBgYAoKPGJyPgoKLS0tLQoKIyMgUHJpbmNpcGFsIENvbXBvbmVudCBBbmFseXNpcyAoUENBKSAKCiMjIyBSdW4gdGhlIFBDQSBhbmQgcHJlcGFyZSBmb3IgcGxvdHRpbmcKCkZpcnN0LCB3ZSBub3JtYWxpemUgdGhlIGNvdW50IGRhdGEgdG8gaGF2ZSBldmVuIHNhbXBsaW5nIGFjcm9zcyBzYW1wbGVzCih3aXRoIHJlc3BlY3QgdG8gbGlicmFyeSBzaXplKSBhbmQgYXBwcm94aW1hdGVseSBldmVuIHZhcmlhbmNlOgoKYGBge3J9CnZzZCA8LSB2YXJpYW5jZVN0YWJpbGl6aW5nVHJhbnNmb3JtYXRpb24oZGRzX3JhdywgYmxpbmQgPSBUUlVFKQpgYGAKCk5leHQsIHdlIHJ1biB0aGUgUENBIGFuZCByZXRyaWV2ZSB0aGUgZGF0YSB0byBwbG90IHdpdGggKmdncGxvdDIqOgoKYGBge3J9CnBjYURhdGEgPC0gcGxvdFBDQSh2c2QsCiAgICAgICAgICAgICAgICAgICBudG9wID0gNTAwLAogICAgICAgICAgICAgICAgICAgaW50Z3JvdXAgPSBjKCJBTUYiLCAiVHJlYXRtZW50IiksCiAgICAgICAgICAgICAgICAgICByZXR1cm5EYXRhID0gVFJVRSkKYGBgCgpXZSBleHRyYWN0IHRoZSBwZXJjZW50YWdlIG9mIHZhcmlhbmNlIGV4cGxhaW5lZCBieSBkaWZmZXJlbnQgcHJpbmNpcGFsIGNvbXBvbmVudHMsCnNvIHdlIGNhbiBsYXRlciBhZGQgdGhpcyBpbmZvcm1hdGlvbiB0byB0aGUgcGxvdDoKCmBgYHtyfQpwZXJjZW50VmFyIDwtIHJvdW5kKDEwMCAqIGF0dHIocGNhRGF0YSwgInBlcmNlbnRWYXIiKSkKcGVyY2VudFZhcgpgYGAKCldlIGNyZWF0ZSBhIHBsb3QgdGl0bGUgd2l0aCB0aGUgc3BlY2llcyBuYW1lIGluIGl0YWxpYyB1c2luZyB0aGUKc29tZXdoYXQgYml6YXJyZSBgZXhwcmVzc2lvbigpYCBmdW5jdGlvbjogCgpgYGB7cn0KcGxvdF90aXRsZSA8LSBleHByZXNzaW9uKCJQQ0Egb2YgIiAqIGl0YWxpYyhHbHljaW5lIH4gbWF4KSAqICIgdHJhbnNjcmlwdCBjb3VudCBkYXRhIikKYGBgCgojIyMgUGxvdCB0aGUgUENBIHJlc3VsdHMKCmBgYHtyfQpnZ3Bsb3QocGNhRGF0YSwKICAgICAgIGFlcyh4ID0gUEMxLCB5ID0gUEMyLCBjb2xvciA9IEFNRiwgc2hhcGUgPSBUcmVhdG1lbnQpKSArCiAgZ2VvbV9wb2ludChzaXplID0gNikgKwogIHhsYWIocGFzdGUwKCJQQzE6ICIsIHBlcmNlbnRWYXJbMV0sICIlIG9mIHZhcmlhbmNlIikpICsKICB5bGFiKHBhc3RlMCgiUEMyOiAiLCBwZXJjZW50VmFyWzJdLCAiJSBvZiB2YXJpYW5jZSIpKSArCiAgZ2d0aXRsZShwbG90X3RpdGxlKQpgYGAKCiMjIyBQbG90IGFnYWluIC0tIHdpdGggc2FtcGxlIG5hbWVzCgpgYGB7cn0KbGlicmFyeShnZ3JlcGVsKQoKcGNhX3Bsb3QyIDwtIGdncGxvdChwY2FEYXRhLAogICAgICAgICAgICAgIGFlcyhQQzEsIFBDMiwgY29sb3IgPSBBTUYsIHNoYXBlID0gVHJlYXRtZW50KSkgKwogIGdlb21fcG9pbnQoc2l6ZSA9IDMpICsKICBnZW9tX2xhYmVsX3JlcGVsKGFlcyhsYWJlbCA9IG5hbWUpKSArCiAgeGxhYihwYXN0ZTAoIlBDMTogIiwgcGVyY2VudFZhclsxXSwgIiUgb2YgdmFyaWFuY2UiKSkgKwogIHlsYWIocGFzdGUwKCJQQzI6ICIsIHBlcmNlbnRWYXJbMl0sICIlIG9mIHZhcmlhbmNlIikpICsKICBnZ3RpdGxlKHBsb3RfdGl0bGUpCgpwcmludChwY2FfcGxvdDIpCmBgYAoKPGJyPgoKLS0tLQoKIyMgREUgYW5hbHlzaXMgLS0gZnVsbCBkYXRhc2V0CgpUaGUgZGVzaWduIGhhcyB0d28gZmFjdG9yczogYEFNRmAgYW5kIGBUcmVhdG1lbnRgLgpSYXRoZXIgdGhhbiBmaXQgYSBtdWx0aXZhcmlhdGUgbW9kZWwsIHdlIGNhbiBzdGFydCBieSBtZXJnaW5nIHRoZSB0d28gaW50byBhCnNpbmdsZSBmYWN0b3IgY2FsbGVkIGBncm91cGAsIGFuZCBmaXQgYSB1bml2YXJpYXRlIG1vZGVsIHdpdGggdGhpcyBmYWN0b3IuCgpgYGB7cn0KZGRzX3JhdyRncm91cCA8LSBmYWN0b3IocGFzdGUoZGRzX3JhdyRBTUYsIGRkc19yYXckVHJlYXRtZW50LCBzZXAgPSAiXyIpKQp0YWJsZShkZHNfcmF3JGdyb3VwKQpgYGAKCldlIHdpbGwgc2V0IHRoZSAicmVmZXJlbmNlIiBsZXZlbCBvZiB0aGUgZmFjdG9yIHRvIGJlIHRoZSBkb3VibGUgbmVnYXRpdmUgY29udHJvbAooZW1wdHkgc3Vic3RyYXRlLCBubyBBZ3JvYmFjdGVyaWEpOgoKYGBge3J9CmRkc19yYXckZ3JvdXAgPC0gcmVsZXZlbChkZHNfcmF3JGdyb3VwLCByZWYgPSAiY29udHJvbF9tb2NrIikKZGRzX3JhdyRncm91cApgYGAKCk5leHQsIHdlIHNldCB0aGUgYW5hbHlzaXMgZGVzaWduOgoKYGBge3J9CmRlc2lnbihkZHNfcmF3KSA8LSBmb3JtdWxhKH4gZ3JvdXApCmBgYAoKQW5kIGZpbmFsbHksIHdlIHBlcmZvcm0gdGhlIGRpZmZlcmVudGlhbCBleHByZXNzaW9uIGFuYWx5c2lzIHdpdGggdGhlIGBERXNlcSgpYApmdW5jdGlvbjoKCmBgYHtyfQpkZHMgPC0gREVTZXEoZGRzX3JhdykKYGBgCgpUaGUgYERFU2VxKClgIGZ1bmN0aW9uIGFib3ZlIHBlcmZvcm1zIHRocmVlIHN0ZXBzIGNvbnNlY3V0aXZlbHk6CiAgCi0gYGVzdGltYXRlU2l6ZUZhY3RvcnMoKWAgLS0gIk5vcm1hbGl6YXRpb24iIGJ5IGxpYnJhcnkgc2l6ZSBhbmQgY29tcG9zaXRpb24uCiAgCiAgTm90ZSB0aGF0ICpERVNlcTIqIGRvZXNu4oCZdCBhY3R1YWxseSBub3JtYWxpemUgdGhlIGNvdW50cyBpbiB0aGUgc2Vuc2UgdGhhdAogIGl0IHByb2R1Y2VzIGEgbWF0cml4IHdpdGggYWRqdXN0ZWQgY291bnRzLgogIEluc3RlYWQgaXQgdXNlcyByYXcgY291bnRzIGFuZCBpbmNsdWRlcyB0aGUgc2l6ZSBmYWN0b3JzIGluIHRoZSBtb2RlbGluZy4KICAKICBUbyBsZWFybiBtb3JlIGFib3V0IGdlbmUgY291bnQgbm9ybWFsaXphdGlvbiwgc2VlCiAgW3RoaXMgdmlkZW9dKGh0dHBzOi8vd3d3LnlvdXR1YmUuY29tL3dhdGNoP3Y9VUZCOTkzeHVmVVUpIGFuZAogIFt0aGlzIHBhZ2VdKGh0dHBzOi8vaGJjdHJhaW5pbmcuZ2l0aHViLmlvL0RHRV93b3Jrc2hvcC9sZXNzb25zLzAyX0RHRV9jb3VudF9ub3JtYWxpemF0aW9uLmh0bWwpLgogIAotIGBlc3RpbWF0ZURpc3BlcnNpb25zKClgIC0tIEVzdGltYXRlIGdlbmUtd2lzZSBkaXNwZXJzaW9uICh2YXJpYW5jZSBpbiBjb3VudHMpLgogIAotIGBuYmlub21XYWxkVGVzdChkZHNPYmopYCAtLSBGaXQgdGhlIG5lZ2F0aXZlIGJpbm9taWFsIEdMTSBhbmQgY2FsY3VsYXRlCiAgV2FsZCBzdGF0aXN0aWNzLCB3aGljaCBpcyB0aGUgdGVzdCBzdGF0aXN0aWMgdW5kZXJseWluZyB0aGUgcC12YWx1ZQogIGZvciB3aGV0aGVyIGEgZ2VuZSBpcyBkaWZmZXJlbnRpYWxseSBleHByZXNzZWQuCgpUaGVzZSBmdW5jdGlvbnMgY291bGQgYWxzbyBiZSBjYWxsZWQgc2VwYXJhdGVseSwKd2hpY2ggd291bGQgYmUgdXNlZnVsIGlmIHlvdSB3YW50IHRvIGJlIGFibGUgdG8gY2hhbmdlIG1vcmUgZGVmYXVsdHMuCgojIyMgVGhlIHJlc3VsdHMgdGFibGUKCmBgYHtyfQpyZXMgPC0gcmVzdWx0cyhkZHMpCmthYmxlKGhlYWQocmVzKSkKYGBgCgpCeSBkZWZhdWx0LCB0aGUgcmVzdWx0cyB0YWJsZSBwcmludHMgc3RhdGlzdGljcwoqKmNvbXBhcmluZyB0aGUgbGFzdCBsZXZlbCBvZiB0aGUgZmFjdG9yIHdpdGggdGhlIGZpcnN0IGxldmVsKio6CnRoYXQgaXMsIGxvZy1mb2xkIGNoYW5nZSBhbmQgcC12YWx1ZXMgZGVzY3JpYmUKZGlmZmVyZW5jZXMgYmV0d2VlbiB0aGVzZSB0d28gbGV2ZWxzIHNwZWNpZmljYWxseS4KSG93ZXZlciwgd2UgY2FuIGVhc2lseSBleHRyYWN0IGVxdWl2YWxlbnQgc3RhdGlzdGljcyBmb3IgYW55IHBhaXJ3aXNlIGNvbXBhcmlzb24KYW1vbmcgb3VyIGZhY3RvciBsZXZlbHMsIHdoaWNoIHdlIHdpbGwgc2VlIGxhdGVyLgoKRm9yIG5vdywgd2Ugd2lsbCBleHBsb3JlIHdoYXQgZWFjaCBjb2x1bW4gaW4gdGhpcyB0YWJsZSBtZWFuczoKCi0gVGhlICoqYGJhc2VNZWFuYCoqIGNvbHVtbiBjb250YWlucyB0aGUgbWVhbiBleHByZXNzaW9uIGxldmVsIGFjcm9zcyBhbGwgc2FtcGxlcy4KCi0gVGhlICoqYGxvZzJGb2xkQ2hhbmdlYCoqIGNvbHVtbiBjb250YWlucyB0aGUgbG9nMi1mb2xkIGNoYW5nZSBvZiBnZW5lIGNvdW50cwogIGJldHdlZW4gdGhlIGNvbXBhcmVkIGxldmVscywgdGhhdCBpcywgaXQgcmVwcmVzZW50cyB0aGUgKmVmZmVjdCBzaXplKi4KICAKICBBIGxvZzItZm9sZCBjaGFuZ2Ugb2YgMSBpbmRpY2F0ZXMgdGhhdCB0aGUgZXhwcmVzc2lvbiBpbiB0aGUgcmVmZXJlbmNlIGxldmVsCiAgaXMgdHdvLWZvbGQgKmxvd2VyKiB0aGFuIHRoYXQgb2YgdGhlIG90aGVyIGxldmVsLAogIGEgbG9nMi1mb2xkIGNoYW5nZSBvZiAyIGluZGljYXRlcyBhIGZvdXItZm9sZCBkaWZmZXJlbmNlLAogIGEgbG9nMi1mb2xkIGNoYW5nZSBvZiAzIGluZGljYXRlcyBhbiBlaWdodC1mb2xkIGRpZmZlcmVuY2UsIGFuZCBzbyBvbi4KICAKICBTaW1pbGFybHksICpuZWdhdGl2ZSBsb2cyLWZvbGQqIHZhbHVlcyBpbmRpY2F0ZSBhIGNoYW5nZSBpbiBnZW5lIGNvdW50cyBpbiB0aGUKICBvdGhlciBkaXJlY3Rpb246IHRoZSByZWZlcmVuY2UgbGV2ZWwgaXMgKmhpZ2hlciogdGhhbiB0aGUgb3RoZXIgbGV2ZWwuCiAgCi0gVGhlIGBsZmNTRWAgY29sdW1uIGluZGljYXRlcyB0aGUgdW5jZXJ0YWludHkgaW4gdGVybXMgb2YgdGhlIHN0YW5kYXJkIGVycm9yCiAgKFNFKSBvZiB0aGUgbG9nMi1mb2xkIGNoYW5nZSBlc3RpbWF0ZS4KICAKLSBUaGUgKipgc3RhdGAqKiBjb2x1bW4gaW5kaWNhdGVzIHRoZSB2YWx1ZSBmb3IgdGhlIFdhbGQgdGVzdCdzIHRlc3Qgc3RhdGlzdGljLgoKLSBUaGUgKipgcHZhbHVlYCoqIGNvbHVtbiByZXBvcnRlZCB0aGUgKnVuY29ycmVjdGVkKiBwLXZhbHVlIGZyb20gdGhlIFdhbGQgdGVzdC4KCi0gQmVjYXVzZSB3ZSBhcmUgdGVzdGluZyBzaWduaWZpY2FuY2UgZm9yICptYW55KiBnZW5lcywKICB3ZSBuZWVkIHRvIGNvcnJlY3QgZm9yIG11bHRpcGxlIHRlc3RpbmcuCiAgREVTZXEyIHVzZXMgdGhlIEJlbmphbWluaS1Ib2NoYmVyZyBGYWxzZSBEaXNjb3ZlcnkgUmF0ZSAoRkRSKSBjb3JyZWN0aW9uLAogIGFuZCB0aGVzZSB2YWx1ZXMgYXJlIHJlcG9ydGVkIGluIHRoZSBjb2x1bW4gKipgcGFkamAqKiAoaS5lLiwgYWRqdXN0ZWQgcC12YWx1ZSkuCgpBIHN1bW1hcnkgb2YgdGhpcyBpbmZvcm1hdGlvbiBhYm91dCBlYWNoIGNvbHVtbiBjYW4gYmUgc2VlbiBieSBydW5uaW5nIHRoZQpgbWNvbHMoKWAgZnVuY3Rpb246CgpgYGB7cn0KbWNvbHMocmVzKQpgYGAKCiMjIyBgTkFgIHZhbHVlcyBpbiB0aGUgcmVzdWx0cyB0YWJsZQoKU29tZSB2YWx1ZXMgaW4gdGhlIHJlc3VsdHMgdGFibGUgY2FuIGJlIHNldCB0byBgTkFgIGZvciBvbmUgb2YgdGhlIGZvbGxvd2luZyByZWFzb25zOgoKLSBJZiBhIGdlbmUgY29udGFpbnMgYSBzYW1wbGUgd2l0aCBhIGNvdW50ICoqb3V0bGllcioqLAogIGJvdGggdGhlIHAtdmFsdWUgYW5kIGFkanVzdGVkIHAtdmFsdWUgd2lsbCBiZSBzZXQgdG8gYE5BYC4KICAoREVTZXEyIHBlcmZvcm1zIG91dGxpZXIgZGV0ZWN0aW9uIHVzaW5nIENvb2sncyBkaXN0YW5jZS4pCiAgCi0gSWYgYWxsIHNhbXBsZXMgaGF2ZSAqKnplcm8gY291bnRzKiogZm9yIGEgZ2l2ZW4gZ2VuZSwKICB0aGUgYGJhc2VNZWFuYCBjb2x1bW4gd2lsbCBiZSB6ZXJvLAogIGFuZCB0aGUgbG9nMi1mb2xkIGNoYW5nZSBlc3RpbWF0ZXMsCiAgcC12YWx1ZSBhbmQgYWRqdXN0ZWQgcC12YWx1ZSB3aWxsIGFsbCBiZSBzZXQgdG8gYE5BYC4KCi0gREVTZXEyIGFsc28gYXV0b21hdGljYWxseSBmaWx0ZXJzIGdlbmVzIHdpdGggYSAqKmxvdyBtZWFuIGNvdW50KioKICBpbiB0aGUgc2Vuc2UgdGhhdCBpdCBkb2VzIG5vdCBpbmNsdWRlIHRoZW0gaW4gdGhlIG11bHRpcGxlIHRlc3RpbmcgY29ycmVjdGlvbi4KICBUaGVyZWZvcmUsIGluIHN1Y2ggY2FzZXMsIHRoZSBwLXZhbHVlIHdpbGwgbm90IGJlIGBOQWAsCiAgYnV0IHRoZSAqYWRqdXN0ZWQqIHAtdmFsdWUgd2lsbCBiZS4KICAKICBCZWNhdXNlIHdlIGhhdmUgdmVyeSBsb3cgcG93ZXIgdG8gZGV0ZWN0IGRpZmZlcmVudGlhbCBleHByZXNzaW9uIGZvciBzdWNoCiAgbG93LWNvdW50IGdlbmVzLCBpdCBpcyBiZW5lZmljaWFsIHRvIHJlbW92ZSB0aGVtIHByaW9yIHRvIHRoZSBtdWx0aXBsZSB0ZXN0aW5nCiAgY29ycmVjdGlvbjogdGhhdCB3YXksIHRoZSBjb3JyZWN0aW9uIGJlY29tZXMgbGVzcyBzZXZlcmUgZm9yIHRoZSByZW1haW5pbmcgZ2VuZXMuCgpMZXQncyBzZWUgaG93IG1hbnkgZ2VuZXMgaGF2ZSBgTkFgIHAtdmFsdWVzOgoKYGBge3J9CiMgTnVtYmVyIG9mIGdlbmVzIHdpdGggTkEgcC12YWx1ZToKc3VtKGlzLm5hKHJlcyRwdmFsdWUpKQoKIyBBcyBhIHByb3BvcnRpb24gb2YgdGhlIHRvdGFsIG51bWJlciBvZiBnZW5lcyBpbiB0aGUgdGVzdDoKc3VtKGlzLm5hKHJlcyRwdmFsdWUpKSAvIG5yb3cocmVzKQpgYGAKCkFuZCBgTkFgIGFkanVzdGVkIHAtdmFsdWVzOgoKYGBge3J9CiMgTnVtYmVyIG9mIGdlbmVzIHdpdGggTkEgcC12YWx1ZToKc3VtKGlzLm5hKHJlcyRwYWRqKSkKCiMgQXMgYSBwcm9wb3J0aW9uIG9mIHRoZSB0b3RhbCBudW1iZXIgb2YgZ2VuZXMgaW4gdGhlIHRlc3Q6CnN1bShpcy5uYShyZXMkcGFkaikpIC8gbnJvdyhyZXMpCmBgYAoKPGJyPgoKLS0tLQoKIyMgREUgYW5hbHlzaXMgLS0gY29udHJhc3QgdHdvIGN1c3RvbSBncm91cHMKClVzaW5nIHRoZSBgcmVzdWx0c05hbWVzYCBmdW5jdGlvbiwKd2UgY2FuIHNlZSB3aGljaCBwYWlyd2lzZSBjb250cmFzdHMgYmV0d2VlbiBkaWZmZXJlbnQgbGV2ZWxzIG9mIHRoZSBmYWN0b3IgYXJlCmF2YWlsYWJsZSAodGhvdWdoIGl0IGlzIG5vdCBkaXNwbGF5ZWQgaW4gYSBwYXJ0aWN1bGFybHkgcmVhZGFibGUgZmFzaGlvbik6CgpgYGB7cn0KcmVzdWx0c05hbWVzKGRkcykKYGBgCgpOb3QgYWxsIHBhaXJ3aXNlIGNvbnRyYXN0cyBiZXR3ZWVuIHRoZSA1IGxldmVscyBpbiBvdXIgYGdyb3VwYCBmYWN0b3IgYXJlIGF2YWlsYWJsZQpoZXJlOiBpbnN0ZWFkLCBgY29udHJvbF9tb2NrYCwgd2hpY2ggd2Ugc2V0IGFzIHRoZSByZWZlcmVuY2UgbGV2ZWwsIGlzIGJlaW5nIGNvbXBhcmVkCndpdGggdGhlIG90aGVyIDMgbGV2ZWxzLgooSG93ZXZlciwgd2UgY2FuIG1ha2Ugb3RoZXIgcGFpcndpc2UgY29tcGFyaXNvbnMsIHRvby4pCgpBYm92ZSwgd2UgbG9va2VkIGF0IHRoZSByZXN1bHRzIGZvciB0aGUgKmxhc3QqIG9mIHRoZXNlIGNvbXBhcmlzb25zCihgZ3JvdXBfUmlfbXliX3ZzX2NvbnRyb2xfbW9ja2AsIGkuZS4gIlJpX215YiIgdnMuICJjb250cm9sX21vY2siKSwKc2ltcGx5IGJlY2F1c2UgREVTZXEyIHdpbGwgc2hvdyB0aGUgbGFzdCBjb21wYXJpc29uIGJ5IGRlZmF1bHQgd2hlbiBjYWxsaW5nCnRoZSBgcmVzdWx0cygpYCBmdW5jdGlvbi4KClRvIHNlZSB0aGUgcmVzdWx0cyB0YWJsZSBmb3Igb25lIG9mIHRoZSBvdGhlciAzIGNvbXBhcmlzb25zLAp3ZSBwYXNzIGEgdmVjdG9yIHRvIHRoZSBgY29udHJhc3RgIGFyZ3VtZW50IG9mIHRoZSBgcmVzdWx0cygpYCBmdW5jdGlvbiB3aXRoCnRoZSBmYWN0b3IgKGBncm91cGApIGFuZCB0aGUgdHdvIGxldmVscyB0byBiZSBjb250cmFzdGVkLgpGb3IgZXhhbXBsZSwgdG8gc2VlIHRoZSByZXN1bHRzIGZvciAiUmlfbW9jayIgdnMuICJjb250cm9sX21vY2siOgoKYGBge3J9CiMgSGVyZSwgd2UgY291bGQgc3BlY2lmeSAqYW55KiBwYWlyd2lzZSBjb250cmFzdCwKIyBub3QganVzdCB0aGUgb25lcyB3aXRoICJjb250cm9sX21vY2siIHRoYXQgcmVzdWx0c05hbWVzKCkgcHJpbnRzIGFzIHNlZW4gYWJvdmUuCm15X2NvbnRyYXN0IDwtIGMoIlJpX21vY2siLCAiY29udHJvbF9tb2NrIikKCnJlcyA8LSByZXN1bHRzKGRkcywKICAgICAgICAgICAgICAgY29udHJhc3QgPSBjKCJncm91cCIsIG15X2NvbnRyYXN0KSkKYGBgCgpIb3cgbWFueSBhZGp1c3RlZCBwLXZhbHVlcyB3ZXJlIGxlc3MgdGhhbiAwLjE/CgpgYGB7cn0Kc3VtKHJlcyRwYWRqIDwgMC4xLCBuYS5ybSA9IFRSVUUpCmBgYAoKV2UnbGwgYWxzbyBjcmVhdGUgYW4gb2JqZWN0IHdpdGggKmFkanVzdGVkIChzaHJ1bmtlbikgTEZDIGVzdGltYXRlcyosCndoaWNoIHdpbGwgYmUgdXNlZnVsIGZvciB2aXN1YWxpemF0aW9uIGFuZCByYW5raW5nIG9mIGdlbmVzIGJ5IExGQzoKCmBgYHtyfQpteV9jb2VmIDwtIHBhc3RlMCgiZ3JvdXBfIiwgcGFzdGUwKG15X2NvbnRyYXN0LCBjb2xsYXBzZSA9ICJfdnNfIikpCm15X2NvZWYKCnJlc19MRkMgPC0gbGZjU2hyaW5rKGRkcywKICAgICAgICAgICAgICAgICAgICAgY29lZiA9IG15X2NvZWYsCiAgICAgICAgICAgICAgICAgICAgIHR5cGUgPSAiYXBlZ2xtIiwKICAgICAgICAgICAgICAgICAgICAgbGZjVGhyZXNob2xkID0gMSkKYGBgCgpIZXJlLCB3ZSBoYWQgdG8gcHJvdmlkZSB0aGUgY29udHJhc3QgKCJjb2VmZmljaWVudCIpIGluIHRoZSBmb3JtYXQKZ2l2ZW4gYnkgYHJlc3VsdHNOYW1lcyhkZHMpYC4KKEFuZCB0aGF0IGxvb2tzIGEgYml0IGNvbmZ1c2luZyBiZWNhdXNlIHRoaXMgZm9ybWF0IHVzZXMgdW5kZXJzY29yZXMgdG8gc2VwYXJhdGUgbGV2ZWxzLAp3aGlsZSBvdXIgZmFjdG9yIGxldmVscyB0aGVtc2VsdmVzIGFsc28gY29udGFpbiB1bmRlcnNjb3Jlcy4pCgpXZSBhbHNvIHNwZWNpZmllZCBhIHRocmVzaG9sZCBvZiAxIGZvciB0aGUgTEZDIHZhbHVlIChgbGZjVGhyZXNob2xkID0gMWApLApzbyB3ZSBnZXQgcy12YWx1ZXMgKGFuYWxvZ291cyB0byBwLXZhbHVlcykgdGhhdCB0ZXN0IG5vdCBmb3IgZGlmZmVyZW50aWFsCmV4cHJlc3Npb24gb2YgYW55IG1hZ25pdHVkZSAoYXMgaW4gdGhlIHRlc3RzIGFib3ZlKSwKYnV0IHdoZXRoZXIgdGhlIExGQyBpcyBncmVhdGVyIHRoYW4gb3VyIHNwZWNpZmllZCB0aHJlc2hvbGQ6IAoKYGBge3J9CmhlYWQocmVzX0xGQykKCnN1bShyZXNfTEZDJHN2YWx1ZSA8IDAuMSwgbmEucm0gPSBUUlVFKQpgYGAKClRoaXMgY2FuIGJlIGEgdXNlZnVsIHdheSB0byB0cnkgYW5kIHRlYXNlIG91dCBzdGF0aXN0aWNhbCBmcm9tIGJpb2xvZ2ljYWwgc2lnbmlmaWNhbmNlLgoKPGJyPgoKLS0tLQoKIyMgVmlzdWFsbHkgZXhwbG9yaW5nIHRoZSByZXN1bHRzCgpXZSB3aWxsIGNyZWF0ZSBhIGZldyBwbG90cywgYnkgd2F5IG9mIGV4YW1wbGUsCm9mIHRoZSByZXN1bHRzIGZvciB0aGUgIlJpX21vY2siIHZlcnN1cyAiY29udHJvbF9tb2NrIiBjb21wYXJpc29uLAp3aGljaCB3ZSBleHRyYWN0ZWQgYWJvdmUuCgojIyMgTUEtcGxvdDoKCkZvciBhIG5pY2Ugb3ZlcnZpZXcgb2YgdGhlIHJlc3VsdHMsIHdlIGNhbiBwbG90IGEgc28tY2FsbGVkICJNQSBwbG90Ii4KQW4gTUEgcGxvdCBzaG93cywgZm9yIGVhY2ggZ2VuZToKCi0gQ291bnQgZGlmZmVyZW5jZXMgaW4gdGVybXMgb2YgTEZDIGJldHdlZW4gdHdvIGdyb3Vwcywgb24gdGhlIHktYXhpcy4KCi0gTWVhbiBjb3VudHMgYWNyb3NzIGJvdGggZ3JvdXBzLCBvbiB0aGUgeC1heGlzLgoKV2UgY2FuIGNyZWF0ZSBhbiBNQSBwbG90IHVzaW5nIERFU2VxMidzIGBwbG90TUFgIGZ1bmN0aW9uLAp3aXRoIHNpZ25pZmljYW50bHkgZGlmZmVyZW50aWFsbHkgZXhwcmVzc2VkIGdlbmVzIGRpc3BsYXllZCBpbiBibHVlOgoKYGBge3J9CnBsb3RNQShyZXMsIHlsaW0gPSBjKC01LCA1KSkKYGBgCgpUbyBiZSBhYmxlIHRvIGN1c3RvbWl6ZSB0aGUgcGxvdCwKd2UnbGwgdXNlIGByZXR1cm5EYXRhID0gVFJVRWAgbGlrZSB3ZSBoYXZlIGRvbmUgd2l0aCBwcmV2aW91cyBwbG90cywKYW5kIHRoZW4gcGxvdCB0aGUgcmVzdWx0aW5nIGRhdGFmcmFtZSB3aXRoICpnZ3Bsb3QyKjoKCmBgYHtyfQpkIDwtIHBsb3RNQShyZXMsIHJldHVybkRhdGEgPSBUUlVFKQoKZ2dwbG90KGQsIGFlcyh4ID0gbWVhbiwgeSA9IGxmYywgY29sb3IgPSBpc0RFKSkgKwogIGdlb21fcG9pbnQoc2l6ZSA9IDAuNSkgKwogIHNjYWxlX3hfbG9nMTAoKSArCiAgc2NhbGVfeV9jb250aW51b3VzKGxpbWl0cyA9IGMoLTEwLCAxMCkpICsKICBzY2FsZV9jb2xvcl9tYW51YWwodmFsdWVzID0gYygnZ3JleTUwJywgJ2JsdWUnKSkgKwogIGd1aWRlcyhjb2xvciA9IEZBTFNFKSArCiAgbGFicyh4ID0gIk1lYW4gb2Ygbm9ybWFsaXplZCBjb3VudHMiLAogICAgICAgeSA9ICJMRkMiKQpgYGAKCldlIGNhbiBzZWUgdGhhdCBsb3dseS1leHByZXNzZWQgZ2VuZXMgdGVuZCB0byBkZXZpYXRlIGZyb20gYW4gTEZDIG9mIDAKKHNhbWUgbWVhbiBleHByZXNzaW9uIGxldmVscyBpbiB0aGUgdHdvIGdyb3VwcykgbXVjaCBtb3JlIHRoYW4KaGlnaGx5LWV4cHJlc3NlZCBnZW5lcyBkby4KSG93ZXZlciwgdGhpcyBpcyBhbiBhcnRpZmFjdCBvZiBub2lzZSBvdmVycG93ZXJpbmcgdGhlIHNpZ25hbCB3aGVuCmV4cHJlc3Npb24gdmFsdWVzIGFyZSBsb3cuCldlIGNhbiBhbHNvIHNlZSB0aGF0IG5vIGdlbmVzIGluIHRoZSBmYXIgbGVmdCBwYXJ0IG9mIHRoZSBwbG90IGFyZQpkaWZmZXJlbnRpYWxseSBleHByZXNzZWQ6IHRoaXMgaXMgZHVlIHRvIHRoaXMgc2FtZSBsYWNrIG9mIHBvd2VyLgoKREVTZXEyIHByb3ZpZGVzIHNldmVyYWwgbWV0aG9kcyB0byBhZGp1c3QgTEZDIGVzdGltYXRlcyBmb3IgdGhpcyBsb3ctZXhwcmVzc2lvbgpiaWFzLgpXZSB1c2VkIG9uZSBvZiB0aG9zZSAoYGxmY1NocmluaygpYCkgYWJvdmUgdG8gcHJvZHVjZSB0aGUgYHJlc19MRkNgIG9iamVjdC4KTGV0J3MgY3JlYXRlIGFub3RoZXIgTUEgcGxvdCB3aXRoIHRoZXNlIGFkanVzdGVkIExGQyBlc3RpbWF0ZXM6CgpgYGB7cn0KZCA8LSBwbG90TUEocmVzX0xGQywgeWxpbSA9IGMoLTUsIDUpLCByZXR1cm5EYXRhID0gVFJVRSkKCmdncGxvdChkLCBhZXMoeCA9IG1lYW4sIHkgPSBsZmMsIGNvbG9yID0gaXNERSkpICsKICBnZW9tX3BvaW50KHNpemUgPSAwLjUpICsKICBzY2FsZV94X2xvZzEwKCkgKwogIHNjYWxlX2NvbG9yX21hbnVhbCh2YWx1ZXMgPSBjKCdncmV5NTAnLCAnYmx1ZScpKSArCiAgZ3VpZGVzKGNvbG9yID0gRkFMU0UpICsKICBsYWJzKHggPSAiTWVhbiBvZiBub3JtYWxpemVkIGNvdW50cyIsCiAgICAgICB5ID0gIlNocnVua2VuIExGQyIpCmBgYAoKTm90ZSB0aGF0IHNpZ25pZmljYW5jZSBpcyBub3cgc3BlY2lmaWVkIGZvciBMRkMgPiAxIC0tIEZJTklTSAoKRmluYWxseSwgZm9yIGEgcGxvdCBsaWtlIHRoaXMsCml0IGNvdWxkIGJlIHVzZWZ1bCB0byBiZSBhYmxlIHRvIGlkZW50aWZ5IGluZGl2aWR1YWwgZ2VuZXMuClRoZXJlIGFyZSB3YXkgdG9vIG1hbnkgdG8gcHJpbnQgdGhlIGdlbmUgbmFtZXMgb24gdGhlIHBsb3QsIHRob3VnaC4KSW5zdGVhZCwgd2UgY2FuIGFsc28gbWFrZSB0aGUgcGxvdCAqKmludGVyYWN0aXZlIHdpdGggUGxvdGx5KiosCnNvIHdlIGNhbiBzZWUgdGhlIGlkZW50aXR5IG9mIGVhY2ggZ2VuZSAqd2hlbiB3ZSBob3ZlciBvdmVyIHRoZSBwb2ludCo6CgpgYGB7cn0KbGlicmFyeShwbG90bHkpCgojIFRvIHNob3cgdGhlIGdlbmUgbmFtZSwgd2UgbmVlZCB0byBoYXZlIGEgY29sdW1uIHdpdGggZ2VuZSBuYW1lcy4KIyBDdXJyZW50bHksIHRoZSBnZW5lIG5hbWVzIGFyZSByb3cgbmFtZXMgYnV0IGdncGxvdDIgKGFuZCBvdGhlciB0aWR5dmVyc2UKIyBhcHBsaWNhdGlvbnMpIGRvbid0IGxpa2UgdGhhdCwgc28gd2UgY3JlYXRlIGEgY29sdW1uIHdpdGggZ2VuZSBuYW1lczoKZCRnZW5lIDwtIHJvd25hbWVzKGQpCgojIEZpcnN0IHdlIGNyZWF0ZSBhIHZlcnkgc2ltaWxhciBnZ3Bsb3QgdG8gd2hhdCB3ZSBkaWQgYWJvdmUsCiMgYnV0IHdlIGFzc2lnbiBnZW5lIG5hbWVzIHRvICJ0ZXh0IjoKcF9tYSA8LSBnZ3Bsb3QoZCwKICAgICAgICAgICAgICAgYWVzKHggPSBtZWFuLCB5ID0gbGZjLCBjb2xvciA9IGlzREUsIHRleHQgPSBnZW5lKSkgKwogIGdlb21fcG9pbnQoc2l6ZSA9IDAuNSkgKwogIHNjYWxlX3hfbG9nMTAoKSArCiAgc2NhbGVfY29sb3JfbWFudWFsKHZhbHVlcyA9IGMoJ2dyZXk1MCcsICdibHVlJykpICsKICBndWlkZXMoY29sb3IgPSBGQUxTRSkgKwogIGxhYnMoeCA9ICJNZWFuIG9mIG5vcm1hbGl6ZWQgY291bnRzIiwKICAgICAgIHkgPSAiU2hydW5rZW4gTEZDIikKCiMgRmluYWxseSwgd2UgbWFrZSB0aGUgcGxvdCBpbnRlcmFjdGl2ZSBhbmQgdGVsbCBQbG90bHkgdGhhdCBpdCBzaG91bGQgc2hvdwojIHRoZSAidGV4dCIgKGkuZS4gZ2VuZSBuYW1lcykgYXMgdGhlICJ0b29sdGlwIiwgbWVhbmluZyB1cG9uIGhvdmVyaW5nOgpnZ3Bsb3RseShwX21hLCB0b29sdGlwID0gInRleHQiKQpgYGAKCiMjIyBQbG90IHNwZWNpZmljIGdlbmVzCgpXZSBjYW4gYWxzbyBjcmVhdGUgcGxvdCBvZiBleHByZXNzaW9uIGxldmVscyBmb3IgaW5kaXZpZHVhbCBnZW5lcy4KVGhhdCBpcyBlc3BlY2lhbGx5IGludGVyZXN0aW5nIGZvciBnZW5lcyB3aXRoIGhpZ2hseSBzaWduaWZpY2FudCBkaWZmZXJlbnRpYWwKZXhwcmVzc2lvbi4KCkxldCdzIHBsb3QgdGhlIHRvcC01IG1vc3Qgc2lnbmlmaWNhbnRseSBkaWZmZXJlbnRpYWxseSBleHByZXNzZWQgZ2VuZXM6CgpgYGB7ciwgZXZhbD1UUlVFfQojIEZpcnN0LCB3ZSBzZWxlY3QgdGhlIDUgZ2VuZXMgd2l0aCB0aGUgbG93ZXN0IGFkanVzdGVkIHAtdmFsdWU6CnRvcDUgPC0gcm93Lm5hbWVzKHJlc1tvcmRlcihyZXMkcGFkailbMTo1XSwgXSkKYGBgCgpgYGB7ciwgZXZhbD1UUlVFfQojIFRoZW4gd2UgY3JlYXRlIGEgZnVuY3Rpb24gdG8gbWFrZSBhIHBsb3QgZm9yIGEgc2luZ2xlIGdlbmU6CnBsb3RnZW5lIDwtIGZ1bmN0aW9uKGdlbmVJRCwgZGRzKSB7CiAgCiAgZCA8LSBwbG90Q291bnRzKGRkcywKICAgICAgICAgICAgICAgICAgZ2VuZSA9IGdlbmVJRCwKICAgICAgICAgICAgICAgICAgaW50Z3JvdXAgPSAiZ3JvdXAiLAogICAgICAgICAgICAgICAgICByZXR1cm5EYXRhID0gVFJVRSkKCiAgcCA8LSBnZ3Bsb3QoZCwgYWVzKHggPSBncm91cCwgeSA9IGNvdW50KSkgKwogICAgICAgICAgZ2VvbV9wb2ludChwb3NpdGlvbiA9IHBvc2l0aW9uX2ppdHRlcih3ID0gMC4xLCBoID0gMCkpICsKICAgICAgICAgIGxhYnModGl0bGUgPSBnZW5lSUQpICsKICAgICAgICAgIHRoZW1lX2J3KCkKICAKICBwcmludChwKQp9CmBgYAoKRmluYWxseSwgd2UgdXNlIGBzYXBwbHkoKWAgdG8gYXBwbHkgdGhpcyBmdW5jdGlvbiB0byBlYWNoIG9mIG91ciBnZW5lcyBpbiB0aGUKYHRvcDVgIHZlY3Rvci4KCmBgYHtyfQpub25lIDwtIHNhcHBseSh0b3A1LCBwbG90Z2VuZSwgZGRzKQpgYGAKCklmIHdlIHdhbnRlZCB0bywgd2UgY291bGQgZWFzaWx5IGNyZWF0ZSBwbG90cyBmb3IgMTAwcyBvZiBnZW5lcywgdGhpcyB3YXkuCgojIyMgSGVhdG1hcAoKV2UgY2FuIGNyZWF0ZSBoZWF0bWFwcyB3aXRoIHRoZSBgcGhlYXRtYXBgIGZ1bmN0aW9uLgpMZXQncyBzdGFydCBieSBjcmVhdGluZyBhIGZ1bmN0aW9uIHRoYXQgd2lsbCBwbG90IGEgaGVhdG1hcCBnaXZlbiBhIHZlY3RvcgpvZiBnZW5lIElEcyBhbmQgYSBERVNlcTIgb2JqZWN0IGBkZHNgOgoKYGBge3IsIGV2YWw9VFJVRX0KcGxvdF9oZWF0bWFwIDwtIGZ1bmN0aW9uKGdlbmVJRHMsIGRkcykgewogIAogIG50ZCA8LSBhc3NheShub3JtVHJhbnNmb3JtKGRkcykpCiAgCiAgbnRkX3NlbCA8LSBudGRbbWF0Y2goZ2VuZUlEcywgcm93bmFtZXMobnRkKSksIF0KICBkZl9tZXRhIDwtIGFzLmRhdGEuZnJhbWUoY29sRGF0YShkZHMpWywgYygiQU1GIiwgIlRyZWF0bWVudCIpXSkKICAKICBwaGVhdG1hcChudGRfc2VsLAogICAgICAgICBjbHVzdGVyX3Jvd3MgPSBGQUxTRSwKICAgICAgICAgY2x1c3Rlcl9jb2xzID0gRkFMU0UsCiAgICAgICAgIHNob3dfcm93bmFtZXMgPSBGQUxTRSwKICAgICAgICAgYW5ub3RhdGlvbl9jb2wgPSBkZl9tZXRhKQp9CmBgYAoKTm93LCB3ZSBjYW4gZWFzaWx5IGNyZWF0ZSBhIGhlYXRtYXAgZm9yIHRoZSB0b3AtMjAgbW9zdCBoaWdobHkgZGlmZmVyZW50aWFsbHkKZXhwcmVzc2VkIGdlbmVzOgoKYGBge3IsIGV2YWw9VFJVRX0KdG9wMjBfREUgPC0gcm93Lm5hbWVzKHJlc1tvcmRlcihyZXMkcGFkailbMToyMF0sIF0pCnBsb3RfaGVhdG1hcCh0b3AyMF9ERSwgZGRzKQpgYGAKCk9yIGZvciB0aGUgMjAgbW9zdCBoaWdobHkgZXhwcmVzc2VkIGdlbmVzOgoKYGBge3IsIGV2YWw9VFJVRX0KdG9wMjBfaGlfaWR4IDwtIG9yZGVyKHJvd01lYW5zKGNvdW50cyhkZHMsIG5vcm1hbGl6ZWQgPSBUUlVFKSksCiAgICAgICAgICAgICAgICAgIGRlY3JlYXNpbmcgPSBUUlVFKVsxOjIwXQp0b3AyMF9oaSA8LSByb3cubmFtZXMoZGRzKVt0b3AyMF9oaV9pZHhdCnBsb3RfaGVhdG1hcCh0b3AyMF9oaSwgZGRzKQpgYGAKCiMjIyBFeHBvcnQgdGhlIHJlc3VsdHMKCkxldCdzIHNhdmUgdGhlIHJlc3VsdHMgZGF0YWZyYW1lIHRvIGZpbGUuCgpOb3RlIHRoYXQgaXQgd2lsbCBvbmx5IGNvbnRhaW4gdGhlIHJlc3VsdHMgZm9yIG9uZSBjb21wYXJpc29uLgpBbHNvLCBpZiB3ZSB3cml0ZSB0aGUgcmVzdWx0cyBkYXRhZnJhbWUgdG8gZmlsZSwKd2Ugd29uJ3QgYmUgYWJsZSB0byB0ZWxsIGZyb20gdGhlIGZpbGUgd2hhdCB0aGUgY29tcGFyaXNvbiBpcywgc28gbGV0J3Mgc3RvcmUKdGhhdCBpbiB0d28gY29sdW1uczoKCmBgYHtyfQpteV9jb250cmFzdAoKbXlfY29udHJhc3RfcGFzdGVkIDwtIHBhc3RlMChteV9jb250cmFzdCwgY29sbGFwc2UgPSAiX3ZzXyIpCm15X2NvbnRyYXN0X3Bhc3RlZApgYGAKCmBgYHtyfQpyZXMkbGV2ZWwxIDwtIG15X2NvbnRyYXN0WzFdCnJlcyRsZXZlbDIgPC0gbXlfY29udHJhc3RbMl0Ka2FibGUoaGVhZChyZXMpKQpgYGAKCk5vdyB3ZSBjYW4gd3JpdGUgYHJlc2AgdG8gZmlsZToKCmBgYHtyLCBldmFsPUZBTFNFfQpyZXNfZmlsZSA8LSBmaWxlLnBhdGgob3V0ZGlyLCBwYXN0ZTAobXlfY29udHJhc3RfcGFzdGVkLCAnX2FsbC1yZXMudHh0JykpCgp3cml0ZS50YWJsZShyZXMsIHJlc19maWxlLAogICAgICAgICAgICBzZXAgPSAnXHQnLCByb3cubmFtZXMgPSBUUlVFLCBxdW90ZSA9IEZBTFNFKQpgYGAKCldlIG1heSBhbHNvIHdhbnQgdG8gc2F2ZSBhIHNlcGFyYXRlIHRhYmxlIHdpdGggb25seSBzaWduaWZpY2FudCByZXN1bHRzOgoKYGBge3IsIGV2YWw9RkFMU0V9CnJlc19zaWdfZmlsZSA8LSBmaWxlLnBhdGgob3V0ZGlyLCBwYXN0ZTAobXlfY29udHJhc3RfcGFzdGVkLCAnX3NpZy1yZXMudHh0JykpCgpyZXNfc2lnIDwtIHJlcyAlPiUKICBhcy5kYXRhLmZyYW1lKCkgJT4lIAogIGRwbHlyOjpmaWx0ZXIocGFkaiA8IDAuMSkKCndyaXRlLnRhYmxlKHJlc19zaWcsIHJlc19zaWdfZmlsZSwKICAgICAgICAgICAgc2VwID0gJ1x0Jywgcm93Lm5hbWVzID0gVFJVRSwgcXVvdGUgPSBGQUxTRSkKYGBgCgo8YnI+CgotLS0tCgojIyBERSBhbmFseXNpcyAtLSBhbGwgcGFpcndpc2UgY29tcGFyaXNvbnMKClRvIHJ1biB0aGUgREUgYW5hbHlzaXMgZm9yIGFsbCBwYWlyd2lzZSBjb21wYXJpc29ucywKd2Ugd2lsbCBzdGFydCBieSB3cml0aW5nIGEgZnVuY3Rpb24gdGhhdCB0YWtlcyBhIGNvbXBhcmlzb24gKGNvbnRyYXN0KSBpbiB0aGUKZm9ybSBvZiBgYyhsZXZlbDEsIGxldmVsMilgIGFsb25nIHdpdGggYSBERVNlcTIgb2JqZWN0IChgZGRzYCksCmFuZCBvdXRwdXQgYSBkYXRhIGZyYW1lIHdpdGggc2lnbmlmaWNhbnRseSBkaWZmZXJlbnRpYWxseSBleHByZXNzZWQgZ2VuZXM6CgpgYGB7cn0Kc2lnX2NvbnRyYXN0IDwtIGZ1bmN0aW9uKG15X2NvbnRyYXN0LCBkZHMpIHsKICAKICByZXNfc2lnIDwtIHJlc3VsdHMoZGRzLAogICAgICAgICAgICAgICAgICAgICBjb250cmFzdCA9IGMoImdyb3VwIiwgbXlfY29udHJhc3QpKSAlPiUKICAgIGFzLmRhdGEuZnJhbWUoKSAlPiUgCiAgICBkcGx5cjo6ZmlsdGVyKHBhZGogPCAwLjEpICU+JQogICAgbXV0YXRlKGxldmVsMSA9IG15X2NvbnRyYXN0WzFdLAogICAgICAgICAgIGxldmVsMiA9IG15X2NvbnRyYXN0WzJdKQogIAogIGNhdChteV9jb250cmFzdFsxXSwgInZlcnN1cyIsIG15X2NvbnRyYXN0WzJdLCAiOiIsIG5yb3cocmVzX3NpZyksICJzaWduaWZpY2FudFxuIikKICAKICByZXR1cm4ocmVzX3NpZykKfQpgYGAKClRoZW4sIHdlIHdpbGwgY3JlYXRlIGEgbGlzdCB3aXRoIGFsbCBwYWlyd2lzZSBjb21iaW5hdGlvbnMgb2Ygb3VyIGBncm91cGAgZmFjdG9yCnVzaW5nIHRoZSBgY29tYm4oKWAgZnVuY3Rpb246CgpgYGB7cn0KY29tcHMgPC0gY29tYm4obGV2ZWxzKGRkc0Bjb2xEYXRhJGdyb3VwKSwgMiwgc2ltcGxpZnkgPSBGQUxTRSkKYGBgCgpGaW5hbGx5LCB3ZSBydW4gdGhlIGZ1bmN0aW9uIGZvciBhbGwgb2Ygb3VyIHBhaXJ3aXNlIGNvbXBhcmlzb25zLAphbmQgdXNpbmcgYGRvLmNhbGwocmJpbmQsIC4uLilgLCB3ZSBjb25jYXRlbmF0ZSBhbGwgdGhlIHJlc3VsdHMgaW4gYSBzaW5nbGUKZGF0YSBmcmFtZToKCmBgYHtyfQpzaWdfYWxsX2NvbnRyYXN0cyA8LSBkby5jYWxsKHJiaW5kLCBsYXBwbHkoY29tcHMsIHNpZ19jb250cmFzdCwgZGRzKSkKCiMgVGhlIGBrYWJsZSgpYCBmdW5jdGlvbiBpcyBqdXN0IHRvIGRpc3BsYXkgdGFibGVzIG5pY2VseSBpbiB0aGlzIFIgTWFya2Rvd24KIyBkb2N1bWVudCAtLSB1c2UgYGhlYWQoc2lnX2FsbF9jb250cmFzdHMpYCBpbiB5b3VyIG93biBjb2RlLgprYWJsZShoZWFkKHNpZ19hbGxfY29udHJhc3RzKSkKYGBgCgotLS0tCgojIyBERSBhbmFseXNpcyAtLSB3aXRoIHR3byBmYWN0b3JzCgojIyMgQ29udHJvbGxpbmcgZm9yIG9uZSBmYWN0b3IKClNheSB3ZSB3YW50ZWQgdG8gYW5hbHl6ZSB0aGUgZWZmZWN0IG9mICJtb2NrIiB2ZXJzdXMgIm15YiIKKCJUcmVhdG1lbnQiIGNvbHVtbikgd2hpbGUgY29udHJvbGxpbmcgZm9yIHRoZSBlZmZlY3RzIG9mICJjb250cm9sIiB2ZXJzdXMgIlJpIgooIkFNRiIgY29sdW1uKS4KCkxldCdzIHN0YXJ0IGJ5IHR1cm5pbmcgIlRyZWF0bWVudCIgYW5kICJBTUYiIGludG8gZmFjdG9ycywKYW5kIHNhdmluZyBhIG5ldyBERVNlcTIgb2JqZWN0OgoKYGBge3J9CiMgQ29udmVydCBUcmVhdG1lbnQgYW5kIEFNRiBpbnRvIGZhY3RvcnM6IApkZHNfcmF3JFRyZWF0bWVudCA8LSByZWxldmVsKGZhY3RvcihkZHNfcmF3JFRyZWF0bWVudCksIHJlZiA9ICJtb2NrIikKZGRzX3JhdyRBTUYgPC0gcmVsZXZlbChmYWN0b3IoZGRzX3JhdyRBTUYpLCByZWYgPSAiY29udHJvbCIpCgojIFNhdmUgYSBuZXcgb2JqZWN0OgpkZHNfMmZfcmF3IDwtIGRkc19yYXcKYGBgCgpUbyBpbmNsdWRlIGJvdGggZmFjdG9ycywgd2UgdXNlIGEgYCtgIGluIHRoZSBmb3JtdWxhLgpOb3RlIHRoYXQgdGhlIG9yZGVyIG1hdHRlcnM6CnVzaW5nIGBBTUYgKyBUcmVhdG1lbnRgLCB3ZSB0ZXN0IGZvciB0aGUgZWZmZWN0IG9mICJUcmVhdG1lbnQiICh0aGUgbGFzdCBmYWN0b3IpLAp3aGlsZSBjb250cm9sbGluZyBmb3IgdGhlIGVmZmVjdCBvZiBBTUYgKHRoZSBmaXJzdCBmYWN0b3IpLgoKYGBge3J9CmRlc2lnbihkZHNfMmZfcmF3KSA8LSBmb3JtdWxhKH4gQU1GICsgVHJlYXRtZW50KQpgYGAKClJ1biBERVNlcSB3aXRoIHRoZSBuZXcgZGVzaWduOgoKYGBge3J9CmRkc18yZiA8LSBERVNlcShkZHNfMmZfcmF3KQpyZXMgPC0gcmVzdWx0cyhkZHNfMmYpCmBgYAoKSG93IG1hbnkgYWRqdXN0ZWQgcC12YWx1ZXMgd2VyZSBsZXNzIHRoYW4gMC4xPwoKYGBge3J9CnN1bShyZXMkcGFkaiA8IDAuMSwgbmEucm0gPSBUUlVFKQpgYGAKCiMjIyBXaXRoIGFuIGludGVyYWN0aW9uIHRlcm0KCldlIGNhbiBhZGQgYW4gaW50ZXJhY3Rpb24gdGVybSB1c2luZyBub3RhdGlvbiBsaWtlIGBsZXZlbDE6bGV2ZWwyYCwgd2l0aCBhIGNvbG9uLgpJbiB0aGlzIGNhc2UsIHdlIHdhbnQgdG8gaW5jbHVkZSBib3RoICJBTUYiLCAiVHJlYXRtZW50IiwgYW5kIHRoZSBpbnRlcmFjdGlvbgpiZXR3ZWVuIHRoZSB0d286CgpgYGB7cn0KIyBTYXZlIGEgbmV3IG9iamVjdDoKZGRzXzJmaV9yYXcgPC0gZGRzX3JhdwoKIyBUaGUgaW50ZXJhY3Rpb24gdGVybSBpcyBgQU1GOlRyZWF0bWVudGA6CmRlc2lnbihkZHNfMmZpX3JhdykgPC0gZm9ybXVsYSh+IEFNRiArIFRyZWF0bWVudCArIEFNRjpUcmVhdG1lbnQpCmBgYAoKUnVuIERFU2VxIHdpdGggdGhlIG5ldyBkZXNpZ246CgpgYGB7cn0KZGRzXzJmaSA8LSBERVNlcShkZHNfMmZpX3JhdykKCnJlc3VsdHNOYW1lcyhkZHNfMmZpKQpgYGAKCkZyb20gdGhlIFtERVNlcTIgdmlnbmV0dGVdKGh0dHA6Ly9iaW9jb25kdWN0b3Iub3JnL3BhY2thZ2VzL2RldmVsL2Jpb2MvdmlnbmV0dGVzL0RFU2VxMi9pbnN0L2RvYy9ERVNlcTIuaHRtbCNpbnRlcmFjdGlvbnMpOgo+VGhlIGtleSBwb2ludCB0byByZW1lbWJlciBhYm91dCBkZXNpZ25zIHdpdGggaW50ZXJhY3Rpb24gdGVybXMgaXMgdGhhdCwgdW5saWtlIGZvciBhIGRlc2lnbiBgfiBnZW5vdHlwZSArIGNvbmRpdGlvbmAsIHdoZXJlIHRoZSBjb25kaXRpb24gZWZmZWN0IHJlcHJlc2VudHMgdGhlIG92ZXJhbGwgZWZmZWN0IGNvbnRyb2xsaW5nIGZvciBkaWZmZXJlbmNlcyBkdWUgdG8gZ2Vub3R5cGUsIGJ5IGFkZGluZyBgZ2Vub3R5cGU6Y29uZGl0aW9uYCwgdGhlIG1haW4gY29uZGl0aW9uIGVmZmVjdCBvbmx5IHJlcHJlc2VudHMgdGhlIGVmZmVjdCBvZiBjb25kaXRpb24gZm9yIHRoZSByZWZlcmVuY2UgbGV2ZWwgb2YgZ2Vub3R5cGUgKEksIG9yIHdoaWNoZXZlciBsZXZlbCB3YXMgZGVmaW5lZCBieSB0aGUgdXNlciBhcyB0aGUgcmVmZXJlbmNlIGxldmVsKS4gVGhlIGludGVyYWN0aW9uIHRlcm1zIGdlbm90eXBlSUkuY29uZGl0aW9uQiBhbmQgZ2Vub3R5cGVJSUkuY29uZGl0aW9uQiBnaXZlIHRoZSBkaWZmZXJlbmNlIGJldHdlZW4gdGhlIGNvbmRpdGlvbiBlZmZlY3QgZm9yIGEgZ2l2ZW4gZ2Vub3R5cGUgYW5kIHRoZSBjb25kaXRpb24gZWZmZWN0IGZvciB0aGUgcmVmZXJlbmNlIGdlbm90eXBlLgoKTm93LCB3ZSBjYW4gY2hlY2sgYSBudW1iZXIgb2YgZGlmZmVyZW50IGVmZmVjdHM6CgojIyMjIERvZXMgdGhlIGVmZmVjdHMgb2YgVHJlYXRtZW50IGRpZmZlciBhbW9uZyBsZXZlbHMgb2YgQU1GPwoKVGhpcyBpcyBkZXNjcmliZWQgYnkgdGhlIGludGVyYWN0aW9uIHRlcm0sIHdoaWNoIGNhbWUgbGFzdCBpbiBvdXIgZm9ybXVsYS4KVGhlcmVmb3JlLCB0aGUgYHJlc3VsdHMoKWAgZnVuY3Rpb24gd2lsbCBnZXQgcmVzdWx0cyBmb3IgdGhlIGludGVyYWN0aW9uIHRlcm0gYnkgZGVmYXVsdDoKCmBgYHtyfQpyZXMgPC0gcmVzdWx0cyhkZHNfMmZpKQprYWJsZShoZWFkKHJlcykpCmBgYAoKIyMjIyBUaGUgZWZmZWN0IG9mIFRyZWF0bWVudCBpbiAiY29udHJvbCIgQU1GOiAKCmBgYHtyfQpyZXMgPC0gcmVzdWx0cyhkZHNfMmZpLAogICAgICAgICAgICAgICBjb250cmFzdCA9IGMoIlRyZWF0bWVudCIsICJteWIiLCAibW9jayIpKQprYWJsZShoZWFkKHJlcykpCmBgYAoKIyMjIyBUaGUgZWZmZWN0IG9mIFRyZWF0bWVudCBpbiAibXliIiBBTUY6IAoKYGBge3J9CnJlcyA8LSByZXN1bHRzKGRkc18yZmksCiAgICAgICAgICAgICAgIGNvbnRyYXN0ID0gbGlzdChjKCJUcmVhdG1lbnRfbXliX3ZzX21vY2siLCAiQU1GUmkuVHJlYXRtZW50bXliIikpKQprYWJsZShoZWFkKHJlcykpCmBgYAoKIyMjIyBUaGUgZWZmZWN0IG9mIEFNRiBpbiB0aGUgIm1vY2siIFRyZWF0bWVudDoKCmBgYHtyfQpyZXMgPC0gcmVzdWx0cyhkZHNfMmZpLAogICAgICAgICAgICAgICAgY29udHJhc3QgPSBjKCJBTUYiLCAiUmkiLCAiY29udHJvbCIpKQprYWJsZShoZWFkKHJlcykpCmBgYAoKIyMjIyBUaGUgZWZmZWN0IG9mIEFNRiBpbiAibXliIiBUcmVhdG1lbnQ6CgpgYGB7cn0KcmVzIDwtIHJlc3VsdHMoZGRzXzJmaSwKICAgICAgICAgICAgICAgY29udHJhc3QgPSBsaXN0KGMoIkFNRl9SaV92c19jb250cm9sIiwgIkFNRlJpLlRyZWF0bWVudG15YiIpKSkKa2FibGUoaGVhZChyZXMpKQpgYGAK