Getting set up
Start an RStudio session at OSC
Show instructions
- Your job should start running pretty soon, and when it’s ready the box should look like this:
- Click
Connect to RStudio Server
at the bottom of the box, and an RStudio Server instance will open. You’re ready to go!
Create an RStudio Project
Show instructions
basedir <- "/fs/project/PAS0471/teach/misc/2021-02_rnaseq/"
# Get your user name (by running a shell command via the `system()` function:
me <- system("echo $USER", intern = TRUE)
# Build the path to the target dir:
proj_dir <- file.path(basedir, me)
# Create the Project:
library(usethis)
create_project(path = proj_dir)
Now, RStudio will reload with the newly created Project open.
If you get the pop-up below, click Don't Save
(do this whenever you get that pop-up):
Load the necessary packages
if(! "pacman" %in% installed.packages()) install.packages("pacman")
packages <- c("DESeq2", # Differential expression analysis
"tidyverse", # Misc. data manipulation and plotting
"here", # Managing file paths
"pheatmap", # Heatmap plot
"apeglm", # For LFC shrinkage
"knitr") # For table printing in Markdown file
pacman::p_load(char = packages)
theme_set(theme_bw()) # Set ggplot theme
Prepare the data
Change the column names, which are very long:
colnames(raw_counts)[7:8]
## [1] "X.fs.scratch.PAS1548.JamesSeq.CAPS.tomato.star_out.X6465_Benitez.PonceM_C6_myb2_V1N_1_S1_L001_R1_001.fastq.gzAligned.sortedByCoord.out.bam"
## [2] "X.fs.scratch.PAS1548.JamesSeq.CAPS.tomato.star_out.X6465_Benitez.PonceM_C6_myb2_V1N_1_S1_L002_R1_001.fastq.gzAligned.sortedByCoord.out.bam"
my_regex <- ".+PonceM_(.+)_V1N.+"
colnames(raw_counts) <- sub(my_regex, "\\1", colnames(raw_counts))
colnames(raw_counts)
## [1] "Geneid" "Chr" "Start" "End" "Strand" "Length" "C6_myb2"
## [8] "C6_myb2" "C6_myb3" "C6_myb3" "C6_myb4" "C6_myb4" "CT_05" "CT_05"
## [15] "CT_06" "CT_06" "CT_09" "CT_09" "T_26" "T_26" "T_29"
## [22] "T_29" "T_30" "T_30" "cnt_1" "cnt_1" "cnt_2" "cnt_2"
## [29] "cnt_3" "cnt_3" "myb_1" "myb_1" "myb_2" "myb_2" "myb_3"
## [36] "myb_3"
Besides the counts, there are columns with metadata for each gene:
## Geneid Chr Start End Strand Length C6_myb2 C6_myb2.1
## 1 Solyc00g160260.1 SL4.0ch00 83863 84177 + 315 0 0
## 2 Solyc00g160270.1 SL4.0ch00 166754 167268 - 515 0 0
## 3 Solyc00g500003.1 SL4.0ch00 311496 382066 - 70571 1 4
## 4 Solyc00g500004.1 SL4.0ch00 417592 418482 + 891 0 0
## 5 Solyc00g500005.1 SL4.0ch00 478389 478640 + 252 0 0
Let’s remove those:
counts <- raw_counts[, 7:ncol(raw_counts)]
rownames(counts) <- raw_counts$Geneid
Replicate samples
In this table, there are two separate entries for each sample: each library was sequenced on two lanes. Recall that in our workflow, we had merged these FASTQ files prior to mapping, but here we are using the table based on the full dataset produced by Matthew.
So, we will go ahead and merge these replicates now, by simply summing the counts:
counts.t <- t(counts)
rownames(counts.t) <- names(raw_counts)[7:36]
sums <- rowsum(counts.t, group = rownames(counts.t))
counts <- t(sums)
(Alternatively, one could use the specialized function DESeq2::collapseReplicates()
for this.)
Check sample IDs
For differential expression analysis, we will be using the popularDESeq2 R/Bioconductor package (paper, website).
We will load both the count table and the metadata into DESeq2. When doing so, DESeq2 assumes that sample IDs in both tables match and are provided in the same order. Let’s make sure this is indeed the case.
Sort both data frames alphabetically:
metadata <- metadata[order(metadata$SampleID), ]
counts <- counts[, order(colnames(counts))]
Check if names are the same:
## [1] "C6_myb2" "C6_myb3" "C6_myb4" "cnt_1" "cnt_2" "cnt_3" "CT_05"
## [8] "CT_06" "CT_09" "myb_1" "myb_2" "myb_3" "T_26" "T_29"
## [15] "T_30"
## [1] "C6_myb2" "C6_myb3" "C6_myb4" "cnt_1" "cnt_2" "cnt_3" "CT_05"
## [8] "CT_06" "CT_09" "myb_1" "myb_2" "myb_3" "T_26" "T_29"
## [15] "T_30"
matching_names <- identical(metadata$SampleID, colnames(counts))
matching_names
## [1] TRUE
if(matching_names == FALSE) stop("Sample ID in metadata and count matrix do not match!")
Create the DESeq2 object
We will create the DESeq2 object using the function DESeqDataSetFromMatrix()
, which we will provide with three pieces of information:
- The count data with argument
countData
.
- The metadata with argument
colData
.
- The model design for the DE analysis – argument
design
.
For now, we will specify ~1
, which means “no design” – we will change this before the actual DE analysis.
dds_raw <- DESeqDataSetFromMatrix(countData = counts,
colData = metadata,
design = ~ 1)
Remove the Ri_noexp
group
#dds_raw <- dds_raw[, -which(dds_raw@colData$group == "Ri_noexp")]
#dds_raw@colData$group <- droplevels(dds_raw@colData$group)
dds_raw <- dds_raw[, -which(dds_raw@colData$Treatment == "noexp")]
#dds_raw@colData$group <- droplevels(dds_raw@colData$group)
Explore the count data
What are number of rows and columns of the count matrix?
## [1] 34688 15
How many genes have non-zero counts?
dim(counts[rowSums(counts) > 0, ])
## [1] 28145 15
How many genes have total counts of at least 10?
dim(counts[rowSums(counts) >= 10, ])
## [1] 24771 15
Histogram of gene counts
Let’s plot a histogram of gene counts:
theme_set(theme_bw())
summed_gene_counts <- data.frame(gene_count = rowSums(counts)) %>%
rownames_to_column("gene_id")
ggplot(data = summed_gene_counts) +
geom_histogram(aes(x = gene_count), binwidth = 10000) +
scale_y_log10(expand = c(0, 0)) +
scale_x_continuous(expand = c(0,0))
Zoom in a bit:
ggplot(data = summed_gene_counts) +
geom_histogram(aes(x = gene_count), binwidth = 1000) +
scale_y_log10(expand = c(0, 0)) +
scale_x_continuous(limits = c(0, 200000), expand = c(0,0)) +
theme(plot.margin = margin(0.5, 0.7, 0.5, 0.5, "cm"))
How are counts distributed across samples? That is, we would like a sum of counts for each column. To get this, we use the apply()
function, which can apply a function (in our case sum()
) to all columns (hence MARGIN = 2
– for rows, use 1
) of our counts
dataframe:
apply(X = counts, MARGIN = 2, FUN = sum)
## C6_myb2 C6_myb3 C6_myb4 cnt_1 cnt_2 cnt_3 CT_05 CT_06
## 18122406 21913320 13659702 18844494 18890045 16634169 22183005 20620412
## CT_09 myb_1 myb_2 myb_3 T_26 T_29 T_30
## 22924853 22070790 22829207 23195471 19902716 23221528 15793928
Principal Component Analysis (PCA)
Run the PCA and prepare for plotting
First, we normalize the count data to have even sampling across samples (with respect to library size) and approximately even variance:
vsd <- varianceStabilizingTransformation(dds_raw, blind = TRUE)
Next, we run the PCA and retrieve the data to plot with ggplot2:
pcaData <- plotPCA(vsd,
ntop = 500,
intgroup = c("AMF", "Treatment"),
returnData = TRUE)
We extract the percentage of variance explained by different principal components, so we can later add this information to the plot:
percentVar <- round(100 * attr(pcaData, "percentVar"))
percentVar
## [1] 58 16
We create a plot title with the species name in italic using the somewhat bizarre expression()
function:
plot_title <- expression("PCA of " * italic(Glycine ~ max) * " transcript count data")
Plot the PCA results
ggplot(pcaData,
aes(x = PC1, y = PC2, color = AMF, shape = Treatment)) +
geom_point(size = 6) +
xlab(paste0("PC1: ", percentVar[1], "% of variance")) +
ylab(paste0("PC2: ", percentVar[2], "% of variance")) +
ggtitle(plot_title)
Plot again – with sample names
library(ggrepel)
pca_plot2 <- ggplot(pcaData,
aes(PC1, PC2, color = AMF, shape = Treatment)) +
geom_point(size = 3) +
geom_label_repel(aes(label = name)) +
xlab(paste0("PC1: ", percentVar[1], "% of variance")) +
ylab(paste0("PC2: ", percentVar[2], "% of variance")) +
ggtitle(plot_title)
print(pca_plot2)
DE analysis – full dataset
The design has two factors: AMF
and Treatment
. Rather than fit a multivariate model, we can start by merging the two into a single factor called group
, and fit a univariate model with this factor.
dds_raw$group <- factor(paste(dds_raw$AMF, dds_raw$Treatment, sep = "_"))
table(dds_raw$group)
##
## control_mock control_myb Ri_mock Ri_myb
## 3 3 3 3
We will set the “reference” level of the factor to be the double negative control (empty substrate, no Agrobacteria):
dds_raw$group <- relevel(dds_raw$group, ref = "control_mock")
dds_raw$group
## [1] control_mock control_mock control_mock Ri_mock Ri_mock
## [6] Ri_mock control_myb control_myb control_myb Ri_myb
## [11] Ri_myb Ri_myb
## Levels: control_mock control_myb Ri_mock Ri_myb
Next, we set the analysis design:
design(dds_raw) <- formula(~ group)
And finally, we perform the differential expression analysis with the DEseq()
function:
The DESeq()
function above performs three steps consecutively:
estimateSizeFactors()
– “Normalization” by library size and composition.
Note that DESeq2 doesn’t actually normalize the counts in the sense that it produces a matrix with adjusted counts. Instead it uses raw counts and includes the size factors in the modeling.
To learn more about gene count normalization, see this video and this page.
estimateDispersions()
– Estimate gene-wise dispersion (variance in counts).
nbinomWaldTest(ddsObj)
– Fit the negative binomial GLM and calculate Wald statistics, which is the test statistic underlying the p-value for whether a gene is differentially expressed.
These functions could also be called separately, which would be useful if you want to be able to change more defaults.
The results table
res <- results(dds)
kable(head(res))
Solyc00g160260.1 |
0.00000 |
NA |
NA |
NA |
NA |
NA |
Solyc00g160270.1 |
0.00000 |
NA |
NA |
NA |
NA |
NA |
Solyc00g500003.1 |
11.11875 |
-0.3323131 |
1.218149 |
-0.2728018 |
0.7850056 |
0.8764988 |
Solyc00g500004.1 |
0.00000 |
NA |
NA |
NA |
NA |
NA |
Solyc00g500005.1 |
0.00000 |
NA |
NA |
NA |
NA |
NA |
Solyc00g500006.1 |
0.00000 |
NA |
NA |
NA |
NA |
NA |
By default, the results table prints statistics comparing the last level of the factor with the first level: that is, log-fold change and p-values describe differences between these two levels specifically. However, we can easily extract equivalent statistics for any pairwise comparison among our factor levels, which we will see later.
For now, we will explore what each column in this table means:
The baseMean
column contains the mean expression level across all samples.
The log2FoldChange
column contains the log2-fold change of gene counts between the compared levels, that is, it represents the effect size.
A log2-fold change of 1 indicates that the expression in the reference level is two-fold lower than that of the other level, a log2-fold change of 2 indicates a four-fold difference, a log2-fold change of 3 indicates an eight-fold difference, and so on.
Similarly, negative log2-fold values indicate a change in gene counts in the other direction: the reference level is higher than the other level.
The lfcSE
column indicates the uncertainty in terms of the standard error (SE) of the log2-fold change estimate.
The stat
column indicates the value for the Wald test’s test statistic.
The pvalue
column reported the uncorrected p-value from the Wald test.
Because we are testing significance for many genes, we need to correct for multiple testing. DESeq2 uses the Benjamini-Hochberg False Discovery Rate (FDR) correction, and these values are reported in the column padj
(i.e., adjusted p-value).
A summary of this information about each column can be seen by running the mcols()
function:
## DataFrame with 6 rows and 2 columns
## type description
## <character> <character>
## baseMean intermediate mean of normalized c..
## log2FoldChange results log2 fold change (ML..
## lfcSE results standard error: grou..
## stat results Wald statistic: grou..
## pvalue results Wald test p-value: g..
## padj results BH adjusted p-values
NA
values in the results table
Some values in the results table can be set to NA
for one of the following reasons:
If a gene contains a sample with a count outlier, both the p-value and adjusted p-value will be set to NA
. (DESeq2 performs outlier detection using Cook’s distance.)
If all samples have zero counts for a given gene, the baseMean
column will be zero, and the log2-fold change estimates, p-value and adjusted p-value will all be set to NA
.
DESeq2 also automatically filters genes with a low mean count in the sense that it does not include them in the multiple testing correction. Therefore, in such cases, the p-value will not be NA
, but the adjusted p-value will be.
Because we have very low power to detect differential expression for such low-count genes, it is beneficial to remove them prior to the multiple testing correction: that way, the correction becomes less severe for the remaining genes.
Let’s see how many genes have NA
p-values:
# Number of genes with NA p-value:
sum(is.na(res$pvalue))
## [1] 6990
# As a proportion of the total number of genes in the test:
sum(is.na(res$pvalue)) / nrow(res)
## [1] 0.2015106
And NA
adjusted p-values:
# Number of genes with NA p-value:
sum(is.na(res$padj))
## [1] 10709
# As a proportion of the total number of genes in the test:
sum(is.na(res$padj)) / nrow(res)
## [1] 0.3087235
DE analysis – contrast two custom groups
Using the resultsNames
function, we can see which pairwise contrasts between different levels of the factor are available (though it is not displayed in a particularly readable fashion):
## [1] "Intercept" "group_control_myb_vs_control_mock"
## [3] "group_Ri_mock_vs_control_mock" "group_Ri_myb_vs_control_mock"
Not all pairwise contrasts between the 5 levels in our group
factor are available here: instead, control_mock
, which we set as the reference level, is being compared with the other 3 levels. (However, we can make other pairwise comparisons, too.)
Above, we looked at the results for the last of these comparisons (group_Ri_myb_vs_control_mock
, i.e. “Ri_myb” vs. “control_mock”), simply because DESeq2 will show the last comparison by default when calling the results()
function.
To see the results table for one of the other 3 comparisons, we pass a vector to the contrast
argument of the results()
function with the factor (group
) and the two levels to be contrasted. For example, to see the results for “Ri_mock” vs. “control_mock”:
# Here, we could specify *any* pairwise contrast,
# not just the ones with "control_mock" that resultsNames() prints as seen above.
my_contrast <- c("Ri_mock", "control_mock")
res <- results(dds,
contrast = c("group", my_contrast))
How many adjusted p-values were less than 0.1?
sum(res$padj < 0.1, na.rm = TRUE)
## [1] 5217
We’ll also create an object with adjusted (shrunken) LFC estimates, which will be useful for visualization and ranking of genes by LFC:
my_coef <- paste0("group_", paste0(my_contrast, collapse = "_vs_"))
my_coef
## [1] "group_Ri_mock_vs_control_mock"
res_LFC <- lfcShrink(dds,
coef = my_coef,
type = "apeglm",
lfcThreshold = 1)
Here, we had to provide the contrast (“coefficient”) in the format given by resultsNames(dds)
. (And that looks a bit confusing because this format uses underscores to separate levels, while our factor levels themselves also contain underscores.)
We also specified a threshold of 1 for the LFC value (lfcThreshold = 1
), so we get s-values (analogous to p-values) that test not for differential expression of any magnitude (as in the tests above), but whether the LFC is greater than our specified threshold:
## log2 fold change (MAP): group Ri mock vs control mock
##
## DataFrame with 6 rows and 4 columns
## baseMean log2FoldChange lfcSE svalue
## <numeric> <numeric> <numeric> <numeric>
## Solyc00g160260.1 0.0000 NA NA NA
## Solyc00g160270.1 0.0000 NA NA NA
## Solyc00g500003.1 11.1187 -0.250441 0.536045 0.576886
## Solyc00g500004.1 0.0000 NA NA NA
## Solyc00g500005.1 0.0000 NA NA NA
## Solyc00g500006.1 0.0000 NA NA NA
sum(res_LFC$svalue < 0.1, na.rm = TRUE)
## [1] 1441
This can be a useful way to try and tease out statistical from biological significance.
Visually exploring the results
We will create a few plots, by way of example, of the results for the “Ri_mock” versus “control_mock” comparison, which we extracted above.
MA-plot:
For a nice overview of the results, we can plot a so-called “MA plot”. An MA plot shows, for each gene:
Count differences in terms of LFC between two groups, on the y-axis.
Mean counts across both groups, on the x-axis.
We can create an MA plot using DESeq2’s plotMA
function, with significantly differentially expressed genes displayed in blue:
plotMA(res, ylim = c(-5, 5))
To be able to customize the plot, we’ll use returnData = TRUE
like we have done with previous plots, and then plot the resulting dataframe with ggplot2:
d <- plotMA(res, returnData = TRUE)
ggplot(d, aes(x = mean, y = lfc, color = isDE)) +
geom_point(size = 0.5) +
scale_x_log10() +
scale_y_continuous(limits = c(-10, 10)) +
scale_color_manual(values = c('grey50', 'blue')) +
guides(color = FALSE) +
labs(x = "Mean of normalized counts",
y = "LFC")
We can see that lowly-expressed genes tend to deviate from an LFC of 0 (same mean expression levels in the two groups) much more than highly-expressed genes do. However, this is an artifact of noise overpowering the signal when expression values are low. We can also see that no genes in the far left part of the plot are differentially expressed: this is due to this same lack of power.
DESeq2 provides several methods to adjust LFC estimates for this low-expression bias. We used one of those (lfcShrink()
) above to produce the res_LFC
object. Let’s create another MA plot with these adjusted LFC estimates:
d <- plotMA(res_LFC, ylim = c(-5, 5), returnData = TRUE)
ggplot(d, aes(x = mean, y = lfc, color = isDE)) +
geom_point(size = 0.5) +
scale_x_log10() +
scale_color_manual(values = c('grey50', 'blue')) +
guides(color = FALSE) +
labs(x = "Mean of normalized counts",
y = "Shrunken LFC")
Note that significance is now specified for LFC > 1 – FINISH
Finally, for a plot like this, it could be useful to be able to identify individual genes. There are way too many to print the gene names on the plot, though. Instead, we can also make the plot interactive with Plotly, so we can see the identity of each gene when we hover over the point:
library(plotly)
# To show the gene name, we need to have a column with gene names.
# Currently, the gene names are row names but ggplot2 (and other tidyverse
# applications) don't like that, so we create a column with gene names:
d$gene <- rownames(d)
# First we create a very similar ggplot to what we did above,
# but we assign gene names to "text":
p_ma <- ggplot(d,
aes(x = mean, y = lfc, color = isDE, text = gene)) +
geom_point(size = 0.5) +
scale_x_log10() +
scale_color_manual(values = c('grey50', 'blue')) +
guides(color = FALSE) +
labs(x = "Mean of normalized counts",
y = "Shrunken LFC")
# Finally, we make the plot interactive and tell Plotly that it should show
# the "text" (i.e. gene names) as the "tooltip", meaning upon hovering:
ggplotly(p_ma, tooltip = "text")
Plot specific genes
We can also create plot of expression levels for individual genes. That is especially interesting for genes with highly significant differential expression.
Let’s plot the top-5 most significantly differentially expressed genes:
# First, we select the 5 genes with the lowest adjusted p-value:
top5 <- row.names(res[order(res$padj)[1:5], ])
# Then we create a function to make a plot for a single gene:
plotgene <- function(geneID, dds) {
d <- plotCounts(dds,
gene = geneID,
intgroup = "group",
returnData = TRUE)
p <- ggplot(d, aes(x = group, y = count)) +
geom_point(position = position_jitter(w = 0.1, h = 0)) +
labs(title = geneID) +
theme_bw()
print(p)
}
Finally, we use sapply()
to apply this function to each of our genes in the top5
vector.
none <- sapply(top5, plotgene, dds)
If we wanted to, we could easily create plots for 100s of genes, this way.
Heatmap
We can create heatmaps with the pheatmap
function. Let’s start by creating a function that will plot a heatmap given a vector of gene IDs and a DESeq2 object dds
:
plot_heatmap <- function(geneIDs, dds) {
ntd <- assay(normTransform(dds))
ntd_sel <- ntd[match(geneIDs, rownames(ntd)), ]
df_meta <- as.data.frame(colData(dds)[, c("AMF", "Treatment")])
pheatmap(ntd_sel,
cluster_rows = FALSE,
cluster_cols = FALSE,
show_rownames = FALSE,
annotation_col = df_meta)
}
Now, we can easily create a heatmap for the top-20 most highly differentially expressed genes:
top20_DE <- row.names(res[order(res$padj)[1:20], ])
plot_heatmap(top20_DE, dds)
Or for the 20 most highly expressed genes:
top20_hi_idx <- order(rowMeans(counts(dds, normalized = TRUE)),
decreasing = TRUE)[1:20]
top20_hi <- row.names(dds)[top20_hi_idx]
plot_heatmap(top20_hi, dds)
Export the results
Let’s save the results dataframe to file.
Note that it will only contain the results for one comparison. Also, if we write the results dataframe to file, we won’t be able to tell from the file what the comparison is, so let’s store that in two columns:
## [1] "Ri_mock" "control_mock"
my_contrast_pasted <- paste0(my_contrast, collapse = "_vs_")
my_contrast_pasted
## [1] "Ri_mock_vs_control_mock"
res$level1 <- my_contrast[1]
res$level2 <- my_contrast[2]
kable(head(res))
Solyc00g160260.1 |
0.00000 |
NA |
NA |
NA |
NA |
NA |
Ri_mock |
control_mock |
Solyc00g160270.1 |
0.00000 |
NA |
NA |
NA |
NA |
NA |
Ri_mock |
control_mock |
Solyc00g500003.1 |
11.11875 |
-1.917985 |
1.255117 |
-1.528132 |
0.1264797 |
0.3040457 |
Ri_mock |
control_mock |
Solyc00g500004.1 |
0.00000 |
NA |
NA |
NA |
NA |
NA |
Ri_mock |
control_mock |
Solyc00g500005.1 |
0.00000 |
NA |
NA |
NA |
NA |
NA |
Ri_mock |
control_mock |
Solyc00g500006.1 |
0.00000 |
NA |
NA |
NA |
NA |
NA |
Ri_mock |
control_mock |
Now we can write res
to file:
res_file <- file.path(outdir, paste0(my_contrast_pasted, '_all-res.txt'))
write.table(res, res_file,
sep = '\t', row.names = TRUE, quote = FALSE)
We may also want to save a separate table with only significant results:
res_sig_file <- file.path(outdir, paste0(my_contrast_pasted, '_sig-res.txt'))
res_sig <- res %>%
as.data.frame() %>%
dplyr::filter(padj < 0.1)
write.table(res_sig, res_sig_file,
sep = '\t', row.names = TRUE, quote = FALSE)
DE analysis – all pairwise comparisons
To run the DE analysis for all pairwise comparisons, we will start by writing a function that takes a comparison (contrast) in the form of c(level1, level2)
along with a DESeq2 object (dds
), and output a data frame with significantly differentially expressed genes:
sig_contrast <- function(my_contrast, dds) {
res_sig <- results(dds,
contrast = c("group", my_contrast)) %>%
as.data.frame() %>%
dplyr::filter(padj < 0.1) %>%
mutate(level1 = my_contrast[1],
level2 = my_contrast[2])
cat(my_contrast[1], "versus", my_contrast[2], ":", nrow(res_sig), "significant\n")
return(res_sig)
}
Then, we will create a list with all pairwise combinations of our group
factor using the combn()
function:
comps <- combn(levels(dds@colData$group), 2, simplify = FALSE)
Finally, we run the function for all of our pairwise comparisons, and using do.call(rbind, ...)
, we concatenate all the results in a single data frame:
sig_all_contrasts <- do.call(rbind, lapply(comps, sig_contrast, dds))
## control_mock versus control_myb : 8327 significant
## control_mock versus Ri_mock : 5217 significant
## control_mock versus Ri_myb : 6109 significant
## control_myb versus Ri_mock : 4223 significant
## control_myb versus Ri_myb : 2263 significant
## Ri_mock versus Ri_myb : 388 significant
# The `kable()` function is just to display tables nicely in this R Markdown
# document -- use `head(sig_all_contrasts)` in your own code.
kable(head(sig_all_contrasts))
61.886942 |
2.682127 |
0.5736071 |
4.675896 |
0.0000029 |
0.0001509 |
control_mock |
control_myb |
3.043385 |
5.914974 |
2.2213307 |
2.662806 |
0.0077492 |
0.0347193 |
control_mock |
control_myb |
2.158579 |
5.117064 |
1.8784057 |
2.724153 |
0.0064467 |
0.0305506 |
control_mock |
control_myb |
4393.208985 |
1.725896 |
0.3936111 |
4.384775 |
0.0000116 |
0.0003991 |
control_mock |
control_myb |
82.440348 |
1.095528 |
0.3450265 |
3.175200 |
0.0014973 |
0.0109290 |
control_mock |
control_myb |
8.675927 |
1.984984 |
0.7470738 |
2.657012 |
0.0078837 |
0.0351581 |
control_mock |
control_myb |
DE analysis – with two factors
Controlling for one factor
Say we wanted to analyze the effect of “mock” versus “myb” (“Treatment” column) while controlling for the effects of “control” versus “Ri” (“AMF” column).
Let’s start by turning “Treatment” and “AMF” into factors, and saving a new DESeq2 object:
# Convert Treatment and AMF into factors:
dds_raw$Treatment <- relevel(factor(dds_raw$Treatment), ref = "mock")
dds_raw$AMF <- relevel(factor(dds_raw$AMF), ref = "control")
# Save a new object:
dds_2f_raw <- dds_raw
To include both factors, we use a +
in the formula. Note that the order matters: using AMF + Treatment
, we test for the effect of “Treatment” (the last factor), while controlling for the effect of AMF (the first factor).
design(dds_2f_raw) <- formula(~ AMF + Treatment)
Run DESeq with the new design:
dds_2f <- DESeq(dds_2f_raw)
res <- results(dds_2f)
How many adjusted p-values were less than 0.1?
sum(res$padj < 0.1, na.rm = TRUE)
## [1] 3719
With an interaction term
We can add an interaction term using notation like level1:level2
, with a colon. In this case, we want to include both “AMF”, “Treatment”, and the interaction between the two:
# Save a new object:
dds_2fi_raw <- dds_raw
# The interaction term is `AMF:Treatment`:
design(dds_2fi_raw) <- formula(~ AMF + Treatment + AMF:Treatment)
Run DESeq with the new design:
dds_2fi <- DESeq(dds_2fi_raw)
resultsNames(dds_2fi)
## [1] "Intercept" "AMF_Ri_vs_control" "Treatment_myb_vs_mock"
## [4] "AMFRi.Treatmentmyb"
From the DESeq2 vignette: >The key point to remember about designs with interaction terms is that, unlike for a design ~ genotype + condition
, where the condition effect represents the overall effect controlling for differences due to genotype, by adding genotype:condition
, the main condition effect only represents the effect of condition for the reference level of genotype (I, or whichever level was defined by the user as the reference level). The interaction terms genotypeII.conditionB and genotypeIII.conditionB give the difference between the condition effect for a given genotype and the condition effect for the reference genotype.
Now, we can check a number of different effects:
Does the effects of Treatment differ among levels of AMF?
This is described by the interaction term, which came last in our formula. Therefore, the results()
function will get results for the interaction term by default:
res <- results(dds_2fi)
kable(head(res))
Solyc00g160260.1 |
0.00000 |
NA |
NA |
NA |
NA |
NA |
Solyc00g160270.1 |
0.00000 |
NA |
NA |
NA |
NA |
NA |
Solyc00g500003.1 |
11.11875 |
1.971865 |
1.747953 |
1.128099 |
0.2592781 |
0.5103138 |
Solyc00g500004.1 |
0.00000 |
NA |
NA |
NA |
NA |
NA |
Solyc00g500005.1 |
0.00000 |
NA |
NA |
NA |
NA |
NA |
Solyc00g500006.1 |
0.00000 |
NA |
NA |
NA |
NA |
NA |
The effect of Treatment in “control” AMF:
res <- results(dds_2fi,
contrast = c("Treatment", "myb", "mock"))
kable(head(res))
Solyc00g160260.1 |
0.00000 |
NA |
NA |
NA |
NA |
NA |
Solyc00g160270.1 |
0.00000 |
NA |
NA |
NA |
NA |
NA |
Solyc00g500003.1 |
11.11875 |
-0.3861962 |
1.215878 |
-0.3176274 |
0.7507676 |
0.8455709 |
Solyc00g500004.1 |
0.00000 |
NA |
NA |
NA |
NA |
NA |
Solyc00g500005.1 |
0.00000 |
NA |
NA |
NA |
NA |
NA |
Solyc00g500006.1 |
0.00000 |
NA |
NA |
NA |
NA |
NA |
The effect of Treatment in “myb” AMF:
res <- results(dds_2fi,
contrast = list(c("Treatment_myb_vs_mock", "AMFRi.Treatmentmyb")))
kable(head(res))
Solyc00g160260.1 |
0.00000 |
NA |
NA |
NA |
NA |
NA |
Solyc00g160270.1 |
0.00000 |
NA |
NA |
NA |
NA |
NA |
Solyc00g500003.1 |
11.11875 |
1.585668 |
1.255784 |
1.262692 |
0.2066998 |
0.9374357 |
Solyc00g500004.1 |
0.00000 |
NA |
NA |
NA |
NA |
NA |
Solyc00g500005.1 |
0.00000 |
NA |
NA |
NA |
NA |
NA |
Solyc00g500006.1 |
0.00000 |
NA |
NA |
NA |
NA |
NA |
The effect of AMF in the “mock” Treatment:
res <- results(dds_2fi,
contrast = c("AMF", "Ri", "control"))
kable(head(res))
Solyc00g160260.1 |
0.00000 |
NA |
NA |
NA |
NA |
NA |
Solyc00g160270.1 |
0.00000 |
NA |
NA |
NA |
NA |
NA |
Solyc00g500003.1 |
11.11875 |
-1.917982 |
1.255115 |
-1.528132 |
0.1264798 |
0.3040459 |
Solyc00g500004.1 |
0.00000 |
NA |
NA |
NA |
NA |
NA |
Solyc00g500005.1 |
0.00000 |
NA |
NA |
NA |
NA |
NA |
Solyc00g500006.1 |
0.00000 |
NA |
NA |
NA |
NA |
NA |
The effect of AMF in “myb” Treatment:
res <- results(dds_2fi,
contrast = list(c("AMF_Ri_vs_control", "AMFRi.Treatmentmyb")))
kable(head(res))
Solyc00g160260.1 |
0.00000 |
NA |
NA |
NA |
NA |
NA |
Solyc00g160270.1 |
0.00000 |
NA |
NA |
NA |
NA |
NA |
Solyc00g500003.1 |
11.11875 |
0.0538826 |
1.216568 |
0.0442907 |
0.9646727 |
0.9859513 |
Solyc00g500004.1 |
0.00000 |
NA |
NA |
NA |
NA |
NA |
Solyc00g500005.1 |
0.00000 |
NA |
NA |
NA |
NA |
NA |
Solyc00g500006.1 |
0.00000 |
NA |
NA |
NA |
NA |
NA |
LS0tCnRpdGxlOiAiRGlmZmVyZW50aWFsIGV4cHJlc3Npb24gKERFKSBhbmFseXNpcyA8YnI+IHdpdGggREVTZXEyIgphdXRob3I6ICJKZWxtZXIgUG9lbHN0cmEiCmRhdGU6ICIzLzI2LzIwMjEiCm91dHB1dDoKICBodG1sX2RvY3VtZW50OgogICAgY29kZV9kb3dubG9hZDogdHJ1ZQogICAgdG9jX2RlcHRoOiAzCiAgICB0b2M6IHRydWUKICAgIHRvY19mbG9hdDogdHJ1ZQogICAgdGhlbWU6IGNlcnVsZWFuCiAgICBoaWdobGlnaHQ6IHRhbmdvCiAgICBhbmNob3Jfc2VjdGlvbnM6IHRydWUKICAgIGRmX3ByaW50OiBrYWJsZQogICAgY3NzOiBodG1sX3BhZ2UuY3NzCmVkaXRvcl9vcHRpb25zOiAKICBjaHVua19vdXRwdXRfdHlwZTogaW5saW5lCi0tLQoKYGBge3Igc2V0dXAsIGluY2x1ZGU9RkFMU0V9CmtuaXRyOjpvcHRzX2NodW5rJHNldCgKICBlY2hvPVRSVUUsIGV2YWw9VFJVRSwgY2FjaGU9VFJVRSwKICB3YXJuaW5nPUZBTFNFLCBtZXNzYWdlPUZBTFNFLAogIGNsYXNzLnNvdXJjZSA9ICJyX2NvZGUiLAogIGNsYXNzLm91dHB1dCA9ICJyX291dHB1dCIsCiAgY2xhc3Mud2FybmluZyA9ICJyX3dhcm5pbmciLAogIGNsYXNzLm1lc3NhZ2UgPSAicl93YXJuaW5nIiwKICBjbGFzcy5lcnJvciA9ICJyX2Vycm9yIgogICkKYGBgCgo8YnI+IDxicj4KCi0tLS0KCiMjIEdldHRpbmcgc2V0IHVwCgojIyMgU3RhcnQgYW4gUlN0dWRpbyBzZXNzaW9uIGF0IE9TQwoKPGRldGFpbHM+PHN1bW1hcnk+U2hvdyBpbnN0cnVjdGlvbnM8L3N1bW1hcnk+CgotIExvZ2luIHRvIE9TQyBhdCA8aHR0cHM6Ly9vbmRlbWFuZC5vc2MuZWR1Pi4KCi0gQ2xpY2sgb24gYEludGVyYWN0aXZlIEFwcHNgICh0b3AgYmFyKSA+IGBSU3R1ZGlvIFNlcnZlciAoT3dlbnMgYW5kIFBpdHplcilgCgo8cCBhbGlnbj0iY2VudGVyIj4KPGltZyBzcmM9aW1nL2FwcHMucG5nIHdpZHRoPSI1MDAiPgo8L3A+CgotIEZpbGwgb3V0IHRoZSBmb3JtIGFzIHNob3duIFtoZXJlXShzbGlkZXMvMDMtT1NDLXNsaWRlcy5odG1sI3JzdHVkaW9fc2VydmVyX2pvYikuCgotIE5vdywgeW91IHNob3VsZCBzZWUgYSBib3ggbGlrZSB0aGlzOgoKPHAgYWxpZ249ImNlbnRlciI+CjxpbWcgc3JjPWltZy9vc2NfcXVldWVkLnBuZyB3aWR0aD0iNzAwIj4KPC9wPgoKLSBZb3VyIGpvYiBzaG91bGQgc3RhcnQgcnVubmluZyBwcmV0dHkgc29vbiwgYW5kIHdoZW4gaXQncyByZWFkeSB0aGUgYm94IHNob3VsZCBsb29rIGxpa2UgdGhpczogCgo8cCBhbGlnbj0iY2VudGVyIj4KPGltZyBzcmM9aW1nL29zY19ydW5uaW5nLnBuZyB3aWR0aD0iNzAwIj4KPC9wPgoKLSBDbGljayBgQ29ubmVjdCB0byBSU3R1ZGlvIFNlcnZlcmAgYXQgdGhlIGJvdHRvbSBvZiB0aGUgYm94LCBhbmQgYW4gUlN0dWRpbyBTZXJ2ZXIgaW5zdGFuY2Ugd2lsbCBvcGVuLiBZb3UncmUgcmVhZHkgdG8gZ28hCgo8L2RldGFpbHM+CgojIyMgQ3JlYXRlIGFuIFJTdHVkaW8gUHJvamVjdAoKPGRldGFpbHM+PHN1bW1hcnk+U2hvdyBpbnN0cnVjdGlvbnM8L3N1bW1hcnk+CgpgYGB7ciwgZXZhbD1GQUxTRX0KYmFzZWRpciA8LSAiL2ZzL3Byb2plY3QvUEFTMDQ3MS90ZWFjaC9taXNjLzIwMjEtMDJfcm5hc2VxLyIKCiMgR2V0IHlvdXIgdXNlciBuYW1lIChieSBydW5uaW5nIGEgc2hlbGwgY29tbWFuZCB2aWEgdGhlIGBzeXN0ZW0oKWAgZnVuY3Rpb246Cm1lIDwtIHN5c3RlbSgiZWNobyAkVVNFUiIsIGludGVybiA9IFRSVUUpCgojIEJ1aWxkIHRoZSBwYXRoIHRvIHRoZSB0YXJnZXQgZGlyOgpwcm9qX2RpciA8LSBmaWxlLnBhdGgoYmFzZWRpciwgbWUpCgojIENyZWF0ZSB0aGUgUHJvamVjdDoKbGlicmFyeSh1c2V0aGlzKQpjcmVhdGVfcHJvamVjdChwYXRoID0gcHJval9kaXIpCmBgYAoKTm93LCBSU3R1ZGlvIHdpbGwgcmVsb2FkIHdpdGggdGhlIG5ld2x5IGNyZWF0ZWQgUHJvamVjdCBvcGVuLgoKSWYgeW91IGdldCB0aGUgcG9wLXVwIGJlbG93LCBjbGljayBgRG9uJ3QgU2F2ZWAgKGRvIHRoaXMgd2hlbmV2ZXIgeW91IGdldCB0aGF0IHBvcC11cCk6Cgo8cCBhbGlnbj0iY2VudGVyIj4KPGltZyBzcmM9aW1nL3JkYXRhLXBvcHVwLnBuZyB3aWR0aD0iMzUwIj4KPC9wPgoKPC9kZXRhaWxzPgoKIyMjIENvcHkgdGhlIGNvdW50IGRhdGEgYW5kIHRoZSBtZXRhZGF0YSBmaWxlcwoKR28gdGhlIGBUZXJtaW5hbGAgdGFiIG5leHQgdG8gdGhlIFIgYENvbnNvbGVgIGluIHRoZSBib3R0b20gbGVmdCBvZiB0aGUgUlN0dWRpbwp3aW5kb3c6CgpgYGB7c2gsIGV2YWw9RkFMU0V9CmNkIC9mcy9wcm9qZWN0L1BBUzA0NzEvdGVhY2gvbWlzYy8yMDIxLTAyX3JuYXNlcS8kVVNFUgoKY3AgLi4vbWFzdGVyL2RhdGEvbWV0YS9tZXRhZGF0YS50eHQgZGF0YS9tZXRhCmNwIC4uL21hc3Rlci9yZXN1bHRzL2NvdW50L2dlbmVfY291bnRzX2FsbC50eHQgcmVzdWx0cy9jb3VudApgYGAKCiMjIyBMb2FkIHRoZSBuZWNlc3NhcnkgcGFja2FnZXMKCmBgYHtyLCB3YXJuaW5nPUZBTFNFLCBtZXNzYWdlPUZBTFNFfQppZighICJwYWNtYW4iICVpbiUgaW5zdGFsbGVkLnBhY2thZ2VzKCkpIGluc3RhbGwucGFja2FnZXMoInBhY21hbiIpCgpwYWNrYWdlcyA8LSBjKCJERVNlcTIiLCAgICAgICAgICAjIERpZmZlcmVudGlhbCBleHByZXNzaW9uIGFuYWx5c2lzCiAgICAgICAgICAgICAgInRpZHl2ZXJzZSIsICAgICAgICMgTWlzYy4gZGF0YSBtYW5pcHVsYXRpb24gYW5kIHBsb3R0aW5nCiAgICAgICAgICAgICAgImhlcmUiLCAgICAgICAgICAgICMgTWFuYWdpbmcgZmlsZSBwYXRocwogICAgICAgICAgICAgICJwaGVhdG1hcCIsICAgICAgICAjIEhlYXRtYXAgcGxvdAogICAgICAgICAgICAgICJhcGVnbG0iLCAgICAgICAgICAjIEZvciBMRkMgc2hyaW5rYWdlCiAgICAgICAgICAgICAgImtuaXRyIikgICAgICAgICAgICMgRm9yIHRhYmxlIHByaW50aW5nIGluIE1hcmtkb3duIGZpbGUKcGFjbWFuOjpwX2xvYWQoY2hhciA9IHBhY2thZ2VzKQoKdGhlbWVfc2V0KHRoZW1lX2J3KCkpICAjIFNldCBnZ3Bsb3QgdGhlbWUKYGBgCgojIyMgSW5wdXQgYW5kIG91dHB1dCBkaXJzIGFuZCBmaWxlcwoKYGBge3IsIGV2YWw9VFJVRSwgZWNobz1GQUxTRX0KY291bnRfdGFibGVfZmlsZSA8LSAicmVzdWx0cy9jb3VudC9nZW5lX2NvdW50c19hbGwudHh0IgptZXRhZGF0YV9maWxlIDwtICJkYXRhL21ldGEvbWV0YWRhdGEudHh0IgoKb3V0ZGlyIDwtICJyZXN1bHRzL0RFLyIKcGxvdGRpciA8LSAicmVzdWx0cy9ERS9maWcvIgpgYGAKCklucHV0IGZpbGVzOgoKLSAqKkdlbmUgY291bnRzIHRhYmxlKiogLS0gdGhlIGZpbGUgZXhhY3RseSBhcyBpdCB3YXMgd3JpdHRlbiBieSAqZmVhdHVyZUNvdW50cyouCiAgCi0gKipNZXRhZGF0YSB0YWJsZSoqIC0tIHNvIHdlIGNhbiBncm91cCBvdXIgc2FtcGxlcyBhbmQgbWFrZSBjb21wYXJpc29ucyBiZXR3ZWVuCiAgdGhlc2UgZ3JvdXBzLgoKYGBge3IsIGV2YWw9RkFMU0V9CmNvdW50X3RhYmxlX2ZpbGUgPC0gaGVyZSgicmVzdWx0cy9jb3VudC9nZW5lX2NvdW50c19hbGwudHh0IikKbWV0YWRhdGFfZmlsZSA8LSBoZXJlKCJkYXRhL21ldGEvbWV0YWRhdGEudHh0IikKYGBgCgpPdXRwdXQgZGlyZWN0b3JpZXMgLS0gYW5kIHdlIGNyZWF0ZSB0aGVtIGlmIHRoZXkgZG9uJ3QgYWxyZWFkeSBleGlzdDoKCmBgYHtyfQpvdXRkaXIgPC0gaGVyZSgicmVzdWx0cy9ERS8iKQpwbG90ZGlyIDwtIGhlcmUoInJlc3VsdHMvREUvZmlnLyIpCgppZiAoIWRpci5leGlzdHMocGxvdGRpcikpIGRpci5jcmVhdGUocGxvdGRpciwgcmVjdXJzaXZlID0gVFJVRSkKaWYgKCFkaXIuZXhpc3RzKHBsb3RkaXIpKSBkaXIuY3JlYXRlKHBsb3RkaXIsIHJlY3Vyc2l2ZSA9IFRSVUUpCmBgYAoKIyMjIExvYWQgaW5wdXQgZGF0YQoKTG9hZCB0aGUgY291bnQgdGFibGUgZnJvbSAqZmVhdHVyZUNvdW50cyo6CgpgYGB7ciwgZXZhbD1UUlVFfQpyYXdfY291bnRzIDwtIHJlYWQudGFibGUoY291bnRfdGFibGVfZmlsZSwKICAgICAgICAgICAgICAgICAgICAgICAgIHNlcCA9ICJcdCIsIGhlYWRlciA9IFRSVUUsIHNraXAgPSAxKQpgYGAKCkxvYWQgdGhlIG1ldGFkYXRhIGluZm9ybWF0aW9uOgoKYGBge3J9Cm1ldGFkYXRhIDwtIHJlYWQudGFibGUobWV0YWRhdGFfZmlsZSwgaGVhZGVyID0gVFJVRSkKCmthYmxlKGhlYWQobWV0YWRhdGEpKQpgYGAKClRoZSBgVHJlYXRtZW50YCBjb2x1bW4gY3VycmVudGx5IGhhcyB0aGUgdmFsdWVzIGBBZ3JvYmFjdGVyaXVtX25vZXhwYCwKYEFncm9iYWN0ZXJpdW1fbXliYCwgYW5kIGBtb2NrYC4KClRvIHNob3J0ZW4gdGhpcyBhIGJpdCwgd2UnbGwgZ2V0IHJpZCBvZiAiQWdyb2JhY3Rlcml1bV8iOgoKYGBge3J9Cm1ldGFkYXRhJFRyZWF0bWVudCA8LSBzdWIoIkFncm9iYWN0ZXJpdW1fIiwgIiIsIG1ldGFkYXRhJFRyZWF0bWVudCkKCnVuaXF1ZShtZXRhZGF0YSRUcmVhdG1lbnQpCmBgYAoKCjxicj4KCi0tLS0KCiMjIFByZXBhcmUgdGhlIGRhdGEKCkNoYW5nZSB0aGUgY29sdW1uIG5hbWVzLCB3aGljaCBhcmUgdmVyeSBsb25nOgoKYGBge3J9CmNvbG5hbWVzKHJhd19jb3VudHMpWzc6OF0KYGBgCgpgYGB7cn0KbXlfcmVnZXggPC0gIi4rUG9uY2VNXyguKylfVjFOLisiCmNvbG5hbWVzKHJhd19jb3VudHMpIDwtIHN1YihteV9yZWdleCwgIlxcMSIsIGNvbG5hbWVzKHJhd19jb3VudHMpKQoKY29sbmFtZXMocmF3X2NvdW50cykKYGBgCgpCZXNpZGVzIHRoZSBjb3VudHMsIHRoZXJlIGFyZSBjb2x1bW5zIHdpdGggbWV0YWRhdGEgZm9yIGVhY2ggZ2VuZToKCmBgYHtyfQpyYXdfY291bnRzWzE6NSwgMTo4XQpgYGAKCkxldCdzIHJlbW92ZSB0aG9zZToKCmBgYHtyfQpjb3VudHMgPC0gcmF3X2NvdW50c1ssIDc6bmNvbChyYXdfY291bnRzKV0Kcm93bmFtZXMoY291bnRzKSA8LSByYXdfY291bnRzJEdlbmVpZApgYGAKCiMjIyBSZXBsaWNhdGUgc2FtcGxlcwoKSW4gdGhpcyB0YWJsZSwgdGhlcmUgYXJlIHR3byBzZXBhcmF0ZSBlbnRyaWVzIGZvciBlYWNoIHNhbXBsZToKZWFjaCBsaWJyYXJ5IHdhcyBzZXF1ZW5jZWQgb24gdHdvIGxhbmVzLgpSZWNhbGwgdGhhdCBpbiBvdXIgd29ya2Zsb3csIHdlIGhhZCBtZXJnZWQgdGhlc2UgRkFTVFEgZmlsZXMgcHJpb3IgdG8gbWFwcGluZywKYnV0IGhlcmUgd2UgYXJlIHVzaW5nIHRoZSB0YWJsZSBiYXNlZCBvbiB0aGUgZnVsbCBkYXRhc2V0IHByb2R1Y2VkIGJ5IE1hdHRoZXcuCgpTbywgd2Ugd2lsbCBnbyBhaGVhZCBhbmQgbWVyZ2UgdGhlc2UgcmVwbGljYXRlcyBub3csIGJ5IHNpbXBseSBzdW1taW5nIHRoZSBjb3VudHM6CgpgYGB7cn0KY291bnRzLnQgPC0gdChjb3VudHMpCnJvd25hbWVzKGNvdW50cy50KSA8LSBuYW1lcyhyYXdfY291bnRzKVs3OjM2XQpzdW1zIDwtIHJvd3N1bShjb3VudHMudCwgZ3JvdXAgPSByb3duYW1lcyhjb3VudHMudCkpCmNvdW50cyA8LSB0KHN1bXMpCmBgYAoKKEFsdGVybmF0aXZlbHksIG9uZSBjb3VsZCB1c2UgdGhlIHNwZWNpYWxpemVkIGZ1bmN0aW9uCmBERVNlcTI6OmNvbGxhcHNlUmVwbGljYXRlcygpYCBmb3IgdGhpcy4pCgojIyMgQ2hlY2sgc2FtcGxlIElEcwoKRm9yIGRpZmZlcmVudGlhbCBleHByZXNzaW9uIGFuYWx5c2lzLCB3ZSB3aWxsIGJlIHVzaW5nIHRoZSBwb3B1bGFyKipERVNlcTIqKgpSL0Jpb2NvbmR1Y3RvciBwYWNrYWdlCihbcGFwZXJdKGh0dHBzOi8vZ2Vub21lYmlvbG9neS5iaW9tZWRjZW50cmFsLmNvbS9hcnRpY2xlcy8xMC4xMTg2L3MxMzA1OS0wMTQtMDU1MC04KSwKW3dlYnNpdGVdKGh0dHBzOi8vYmlvY29uZHVjdG9yLm9yZy9wYWNrYWdlcy9yZWxlYXNlL2Jpb2MvaHRtbC9ERVNlcTIuaHRtbCkpLgoKV2Ugd2lsbCBsb2FkIGJvdGggdGhlIGNvdW50IHRhYmxlIGFuZCB0aGUgbWV0YWRhdGEgaW50byAqREVTZXEyKi4KV2hlbiBkb2luZyBzbywgKkRFU2VxMiogYXNzdW1lcyB0aGF0IHNhbXBsZSBJRHMgaW4gYm90aCB0YWJsZXMgbWF0Y2ggYW5kIAphcmUgcHJvdmlkZWQgaW4gdGhlIHNhbWUgb3JkZXIuIExldCdzIG1ha2Ugc3VyZSB0aGlzIGlzIGluZGVlZCB0aGUgY2FzZS4KClNvcnQgYm90aCBkYXRhIGZyYW1lcyBhbHBoYWJldGljYWxseToKCmBgYHtyfQptZXRhZGF0YSA8LSBtZXRhZGF0YVtvcmRlcihtZXRhZGF0YSRTYW1wbGVJRCksIF0KY291bnRzIDwtIGNvdW50c1ssIG9yZGVyKGNvbG5hbWVzKGNvdW50cykpXQpgYGAKCkNoZWNrIGlmIG5hbWVzIGFyZSB0aGUgc2FtZToKCmBgYHtyfQptZXRhZGF0YSRTYW1wbGVJRAoKY29sbmFtZXMoY291bnRzKQoKbWF0Y2hpbmdfbmFtZXMgPC0gaWRlbnRpY2FsKG1ldGFkYXRhJFNhbXBsZUlELCBjb2xuYW1lcyhjb3VudHMpKQptYXRjaGluZ19uYW1lcwppZihtYXRjaGluZ19uYW1lcyA9PSBGQUxTRSkgc3RvcCgiU2FtcGxlIElEIGluIG1ldGFkYXRhIGFuZCBjb3VudCBtYXRyaXggZG8gbm90IG1hdGNoISIpCmBgYAoKIyMjIENyZWF0ZSB0aGUgREVTZXEyIG9iamVjdAoKV2Ugd2lsbCBjcmVhdGUgdGhlIERFU2VxMiBvYmplY3QgdXNpbmcgdGhlIGZ1bmN0aW9uIGBERVNlcURhdGFTZXRGcm9tTWF0cml4KClgLAp3aGljaCB3ZSB3aWxsIHByb3ZpZGUgd2l0aCB0aHJlZSBwaWVjZXMgb2YgaW5mb3JtYXRpb246CgotIFRoZSBjb3VudCBkYXRhIHdpdGggYXJndW1lbnQgYGNvdW50RGF0YWAuCi0gVGhlIG1ldGFkYXRhIHdpdGggYXJndW1lbnQgYGNvbERhdGFgLgotIFRoZSBtb2RlbCBkZXNpZ24gZm9yIHRoZSBERSBhbmFseXNpcyAtLSBhcmd1bWVudCBgZGVzaWduYC4gIAogIEZvciBub3csIHdlIHdpbGwgc3BlY2lmeSBgfjFgLCB3aGljaCBtZWFucyAibm8gZGVzaWduIiAtLQogIHdlIHdpbGwgY2hhbmdlIHRoaXMgYmVmb3JlIHRoZSBhY3R1YWwgREUgYW5hbHlzaXMuCgpgYGB7cn0KZGRzX3JhdyA8LSBERVNlcURhdGFTZXRGcm9tTWF0cml4KGNvdW50RGF0YSA9IGNvdW50cywKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGNvbERhdGEgPSBtZXRhZGF0YSwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGRlc2lnbiA9IH4gMSkKYGBgCgojIyMgUmVtb3ZlIHRoZSBgUmlfbm9leHBgIGdyb3VwCgpgYGB7cn0KI2Rkc19yYXcgPC0gZGRzX3Jhd1ssIC13aGljaChkZHNfcmF3QGNvbERhdGEkZ3JvdXAgPT0gIlJpX25vZXhwIildCiNkZHNfcmF3QGNvbERhdGEkZ3JvdXAgPC0gZHJvcGxldmVscyhkZHNfcmF3QGNvbERhdGEkZ3JvdXApCgpkZHNfcmF3IDwtIGRkc19yYXdbLCAtd2hpY2goZGRzX3Jhd0Bjb2xEYXRhJFRyZWF0bWVudCA9PSAibm9leHAiKV0KI2Rkc19yYXdAY29sRGF0YSRncm91cCA8LSBkcm9wbGV2ZWxzKGRkc19yYXdAY29sRGF0YSRncm91cCkKYGBgCgo8YnI+CgotLS0tCgojIyBFeHBsb3JlIHRoZSBjb3VudCBkYXRhCgpXaGF0IGFyZSBudW1iZXIgb2Ygcm93cyBhbmQgY29sdW1ucyBvZiB0aGUgY291bnQgbWF0cml4PwoKYGBge3J9CmRpbShjb3VudHMpCmBgYAoKSG93IG1hbnkgZ2VuZXMgaGF2ZSBub24temVybyBjb3VudHM/CgpgYGB7cn0KZGltKGNvdW50c1tyb3dTdW1zKGNvdW50cykgPiAwLCBdKQoKYGBgCgpIb3cgbWFueSBnZW5lcyBoYXZlIHRvdGFsIGNvdW50cyBvZiBhdCBsZWFzdCAxMD8KCmBgYHtyfQpkaW0oY291bnRzW3Jvd1N1bXMoY291bnRzKSA+PSAxMCwgXSkKYGBgCgojIyMgSGlzdG9ncmFtIG9mIGdlbmUgY291bnRzCgpMZXQncyBwbG90IGEgaGlzdG9ncmFtIG9mIGdlbmUgY291bnRzOgoKYGBge3J9CnRoZW1lX3NldCh0aGVtZV9idygpKQoKc3VtbWVkX2dlbmVfY291bnRzIDwtIGRhdGEuZnJhbWUoZ2VuZV9jb3VudCA9IHJvd1N1bXMoY291bnRzKSkgJT4lCiAgcm93bmFtZXNfdG9fY29sdW1uKCJnZW5lX2lkIikKCmdncGxvdChkYXRhID0gc3VtbWVkX2dlbmVfY291bnRzKSArCiAgZ2VvbV9oaXN0b2dyYW0oYWVzKHggPSBnZW5lX2NvdW50KSwgYmlud2lkdGggPSAxMDAwMCkgKwogIHNjYWxlX3lfbG9nMTAoZXhwYW5kID0gYygwLCAwKSkgKwogIHNjYWxlX3hfY29udGludW91cyhleHBhbmQgPSBjKDAsMCkpCmBgYAoKWm9vbSBpbiBhIGJpdDoKCmBgYHtyfQpnZ3Bsb3QoZGF0YSA9IHN1bW1lZF9nZW5lX2NvdW50cykgKwogIGdlb21faGlzdG9ncmFtKGFlcyh4ID0gZ2VuZV9jb3VudCksIGJpbndpZHRoID0gMTAwMCkgKwogIHNjYWxlX3lfbG9nMTAoZXhwYW5kID0gYygwLCAwKSkgKwogIHNjYWxlX3hfY29udGludW91cyhsaW1pdHMgPSBjKDAsIDIwMDAwMCksIGV4cGFuZCA9IGMoMCwwKSkgKwogIHRoZW1lKHBsb3QubWFyZ2luID0gbWFyZ2luKDAuNSwgMC43LCAwLjUsIDAuNSwgImNtIikpCmBgYAoKSG93IGFyZSBjb3VudHMgZGlzdHJpYnV0ZWQgYWNyb3NzIHNhbXBsZXM/ClRoYXQgaXMsIHdlIHdvdWxkIGxpa2UgYSBzdW0gb2YgY291bnRzIGZvciBlYWNoIGNvbHVtbi4KVG8gZ2V0IHRoaXMsIHdlIHVzZSB0aGUgYGFwcGx5KClgIGZ1bmN0aW9uLCB3aGljaCBjYW4gYXBwbHkgYSBmdW5jdGlvbgooaW4gb3VyIGNhc2UgYHN1bSgpYCkgdG8gYWxsIGNvbHVtbnMgKGhlbmNlIGBNQVJHSU4gPSAyYCAtLSBmb3Igcm93cywgdXNlIGAxYCkKb2Ygb3VyIGBjb3VudHNgIGRhdGFmcmFtZToKCmBgYHtyfQphcHBseShYID0gY291bnRzLCBNQVJHSU4gPSAyLCBGVU4gPSBzdW0pCmBgYAoKPGJyPgoKLS0tLQoKIyMgUHJpbmNpcGFsIENvbXBvbmVudCBBbmFseXNpcyAoUENBKSAKCiMjIyBSdW4gdGhlIFBDQSBhbmQgcHJlcGFyZSBmb3IgcGxvdHRpbmcKCkZpcnN0LCB3ZSBub3JtYWxpemUgdGhlIGNvdW50IGRhdGEgdG8gaGF2ZSBldmVuIHNhbXBsaW5nIGFjcm9zcyBzYW1wbGVzCih3aXRoIHJlc3BlY3QgdG8gbGlicmFyeSBzaXplKSBhbmQgYXBwcm94aW1hdGVseSBldmVuIHZhcmlhbmNlOgoKYGBge3J9CnZzZCA8LSB2YXJpYW5jZVN0YWJpbGl6aW5nVHJhbnNmb3JtYXRpb24oZGRzX3JhdywgYmxpbmQgPSBUUlVFKQpgYGAKCk5leHQsIHdlIHJ1biB0aGUgUENBIGFuZCByZXRyaWV2ZSB0aGUgZGF0YSB0byBwbG90IHdpdGggKmdncGxvdDIqOgoKYGBge3J9CnBjYURhdGEgPC0gcGxvdFBDQSh2c2QsCiAgICAgICAgICAgICAgICAgICBudG9wID0gNTAwLAogICAgICAgICAgICAgICAgICAgaW50Z3JvdXAgPSBjKCJBTUYiLCAiVHJlYXRtZW50IiksCiAgICAgICAgICAgICAgICAgICByZXR1cm5EYXRhID0gVFJVRSkKYGBgCgpXZSBleHRyYWN0IHRoZSBwZXJjZW50YWdlIG9mIHZhcmlhbmNlIGV4cGxhaW5lZCBieSBkaWZmZXJlbnQgcHJpbmNpcGFsIGNvbXBvbmVudHMsCnNvIHdlIGNhbiBsYXRlciBhZGQgdGhpcyBpbmZvcm1hdGlvbiB0byB0aGUgcGxvdDoKCmBgYHtyfQpwZXJjZW50VmFyIDwtIHJvdW5kKDEwMCAqIGF0dHIocGNhRGF0YSwgInBlcmNlbnRWYXIiKSkKcGVyY2VudFZhcgpgYGAKCldlIGNyZWF0ZSBhIHBsb3QgdGl0bGUgd2l0aCB0aGUgc3BlY2llcyBuYW1lIGluIGl0YWxpYyB1c2luZyB0aGUKc29tZXdoYXQgYml6YXJyZSBgZXhwcmVzc2lvbigpYCBmdW5jdGlvbjogCgpgYGB7cn0KcGxvdF90aXRsZSA8LSBleHByZXNzaW9uKCJQQ0Egb2YgIiAqIGl0YWxpYyhHbHljaW5lIH4gbWF4KSAqICIgdHJhbnNjcmlwdCBjb3VudCBkYXRhIikKYGBgCgojIyMgUGxvdCB0aGUgUENBIHJlc3VsdHMKCmBgYHtyfQpnZ3Bsb3QocGNhRGF0YSwKICAgICAgIGFlcyh4ID0gUEMxLCB5ID0gUEMyLCBjb2xvciA9IEFNRiwgc2hhcGUgPSBUcmVhdG1lbnQpKSArCiAgZ2VvbV9wb2ludChzaXplID0gNikgKwogIHhsYWIocGFzdGUwKCJQQzE6ICIsIHBlcmNlbnRWYXJbMV0sICIlIG9mIHZhcmlhbmNlIikpICsKICB5bGFiKHBhc3RlMCgiUEMyOiAiLCBwZXJjZW50VmFyWzJdLCAiJSBvZiB2YXJpYW5jZSIpKSArCiAgZ2d0aXRsZShwbG90X3RpdGxlKQpgYGAKCiMjIyBQbG90IGFnYWluIC0tIHdpdGggc2FtcGxlIG5hbWVzCgpgYGB7cn0KbGlicmFyeShnZ3JlcGVsKQoKcGNhX3Bsb3QyIDwtIGdncGxvdChwY2FEYXRhLAogICAgICAgICAgICAgIGFlcyhQQzEsIFBDMiwgY29sb3IgPSBBTUYsIHNoYXBlID0gVHJlYXRtZW50KSkgKwogIGdlb21fcG9pbnQoc2l6ZSA9IDMpICsKICBnZW9tX2xhYmVsX3JlcGVsKGFlcyhsYWJlbCA9IG5hbWUpKSArCiAgeGxhYihwYXN0ZTAoIlBDMTogIiwgcGVyY2VudFZhclsxXSwgIiUgb2YgdmFyaWFuY2UiKSkgKwogIHlsYWIocGFzdGUwKCJQQzI6ICIsIHBlcmNlbnRWYXJbMl0sICIlIG9mIHZhcmlhbmNlIikpICsKICBnZ3RpdGxlKHBsb3RfdGl0bGUpCgpwcmludChwY2FfcGxvdDIpCmBgYAoKPGJyPgoKLS0tLQoKIyMgREUgYW5hbHlzaXMgLS0gZnVsbCBkYXRhc2V0CgpUaGUgZGVzaWduIGhhcyB0d28gZmFjdG9yczogYEFNRmAgYW5kIGBUcmVhdG1lbnRgLgpSYXRoZXIgdGhhbiBmaXQgYSBtdWx0aXZhcmlhdGUgbW9kZWwsIHdlIGNhbiBzdGFydCBieSBtZXJnaW5nIHRoZSB0d28gaW50byBhCnNpbmdsZSBmYWN0b3IgY2FsbGVkIGBncm91cGAsIGFuZCBmaXQgYSB1bml2YXJpYXRlIG1vZGVsIHdpdGggdGhpcyBmYWN0b3IuCgpgYGB7cn0KZGRzX3JhdyRncm91cCA8LSBmYWN0b3IocGFzdGUoZGRzX3JhdyRBTUYsIGRkc19yYXckVHJlYXRtZW50LCBzZXAgPSAiXyIpKQp0YWJsZShkZHNfcmF3JGdyb3VwKQpgYGAKCldlIHdpbGwgc2V0IHRoZSAicmVmZXJlbmNlIiBsZXZlbCBvZiB0aGUgZmFjdG9yIHRvIGJlIHRoZSBkb3VibGUgbmVnYXRpdmUgY29udHJvbAooZW1wdHkgc3Vic3RyYXRlLCBubyBBZ3JvYmFjdGVyaWEpOgoKYGBge3J9CmRkc19yYXckZ3JvdXAgPC0gcmVsZXZlbChkZHNfcmF3JGdyb3VwLCByZWYgPSAiY29udHJvbF9tb2NrIikKZGRzX3JhdyRncm91cApgYGAKCk5leHQsIHdlIHNldCB0aGUgYW5hbHlzaXMgZGVzaWduOgoKYGBge3J9CmRlc2lnbihkZHNfcmF3KSA8LSBmb3JtdWxhKH4gZ3JvdXApCmBgYAoKQW5kIGZpbmFsbHksIHdlIHBlcmZvcm0gdGhlIGRpZmZlcmVudGlhbCBleHByZXNzaW9uIGFuYWx5c2lzIHdpdGggdGhlIGBERXNlcSgpYApmdW5jdGlvbjoKCmBgYHtyfQpkZHMgPC0gREVTZXEoZGRzX3JhdykKYGBgCgpUaGUgYERFU2VxKClgIGZ1bmN0aW9uIGFib3ZlIHBlcmZvcm1zIHRocmVlIHN0ZXBzIGNvbnNlY3V0aXZlbHk6CiAgCi0gYGVzdGltYXRlU2l6ZUZhY3RvcnMoKWAgLS0gIk5vcm1hbGl6YXRpb24iIGJ5IGxpYnJhcnkgc2l6ZSBhbmQgY29tcG9zaXRpb24uCiAgCiAgTm90ZSB0aGF0ICpERVNlcTIqIGRvZXNu4oCZdCBhY3R1YWxseSBub3JtYWxpemUgdGhlIGNvdW50cyBpbiB0aGUgc2Vuc2UgdGhhdAogIGl0IHByb2R1Y2VzIGEgbWF0cml4IHdpdGggYWRqdXN0ZWQgY291bnRzLgogIEluc3RlYWQgaXQgdXNlcyByYXcgY291bnRzIGFuZCBpbmNsdWRlcyB0aGUgc2l6ZSBmYWN0b3JzIGluIHRoZSBtb2RlbGluZy4KICAKICBUbyBsZWFybiBtb3JlIGFib3V0IGdlbmUgY291bnQgbm9ybWFsaXphdGlvbiwgc2VlCiAgW3RoaXMgdmlkZW9dKGh0dHBzOi8vd3d3LnlvdXR1YmUuY29tL3dhdGNoP3Y9VUZCOTkzeHVmVVUpIGFuZAogIFt0aGlzIHBhZ2VdKGh0dHBzOi8vaGJjdHJhaW5pbmcuZ2l0aHViLmlvL0RHRV93b3Jrc2hvcC9sZXNzb25zLzAyX0RHRV9jb3VudF9ub3JtYWxpemF0aW9uLmh0bWwpLgogIAotIGBlc3RpbWF0ZURpc3BlcnNpb25zKClgIC0tIEVzdGltYXRlIGdlbmUtd2lzZSBkaXNwZXJzaW9uICh2YXJpYW5jZSBpbiBjb3VudHMpLgogIAotIGBuYmlub21XYWxkVGVzdChkZHNPYmopYCAtLSBGaXQgdGhlIG5lZ2F0aXZlIGJpbm9taWFsIEdMTSBhbmQgY2FsY3VsYXRlCiAgV2FsZCBzdGF0aXN0aWNzLCB3aGljaCBpcyB0aGUgdGVzdCBzdGF0aXN0aWMgdW5kZXJseWluZyB0aGUgcC12YWx1ZQogIGZvciB3aGV0aGVyIGEgZ2VuZSBpcyBkaWZmZXJlbnRpYWxseSBleHByZXNzZWQuCgpUaGVzZSBmdW5jdGlvbnMgY291bGQgYWxzbyBiZSBjYWxsZWQgc2VwYXJhdGVseSwKd2hpY2ggd291bGQgYmUgdXNlZnVsIGlmIHlvdSB3YW50IHRvIGJlIGFibGUgdG8gY2hhbmdlIG1vcmUgZGVmYXVsdHMuCgojIyMgVGhlIHJlc3VsdHMgdGFibGUKCmBgYHtyfQpyZXMgPC0gcmVzdWx0cyhkZHMpCmthYmxlKGhlYWQocmVzKSkKYGBgCgpCeSBkZWZhdWx0LCB0aGUgcmVzdWx0cyB0YWJsZSBwcmludHMgc3RhdGlzdGljcwoqKmNvbXBhcmluZyB0aGUgbGFzdCBsZXZlbCBvZiB0aGUgZmFjdG9yIHdpdGggdGhlIGZpcnN0IGxldmVsKio6CnRoYXQgaXMsIGxvZy1mb2xkIGNoYW5nZSBhbmQgcC12YWx1ZXMgZGVzY3JpYmUKZGlmZmVyZW5jZXMgYmV0d2VlbiB0aGVzZSB0d28gbGV2ZWxzIHNwZWNpZmljYWxseS4KSG93ZXZlciwgd2UgY2FuIGVhc2lseSBleHRyYWN0IGVxdWl2YWxlbnQgc3RhdGlzdGljcyBmb3IgYW55IHBhaXJ3aXNlIGNvbXBhcmlzb24KYW1vbmcgb3VyIGZhY3RvciBsZXZlbHMsIHdoaWNoIHdlIHdpbGwgc2VlIGxhdGVyLgoKRm9yIG5vdywgd2Ugd2lsbCBleHBsb3JlIHdoYXQgZWFjaCBjb2x1bW4gaW4gdGhpcyB0YWJsZSBtZWFuczoKCi0gVGhlICoqYGJhc2VNZWFuYCoqIGNvbHVtbiBjb250YWlucyB0aGUgbWVhbiBleHByZXNzaW9uIGxldmVsIGFjcm9zcyBhbGwgc2FtcGxlcy4KCi0gVGhlICoqYGxvZzJGb2xkQ2hhbmdlYCoqIGNvbHVtbiBjb250YWlucyB0aGUgbG9nMi1mb2xkIGNoYW5nZSBvZiBnZW5lIGNvdW50cwogIGJldHdlZW4gdGhlIGNvbXBhcmVkIGxldmVscywgdGhhdCBpcywgaXQgcmVwcmVzZW50cyB0aGUgKmVmZmVjdCBzaXplKi4KICAKICBBIGxvZzItZm9sZCBjaGFuZ2Ugb2YgMSBpbmRpY2F0ZXMgdGhhdCB0aGUgZXhwcmVzc2lvbiBpbiB0aGUgcmVmZXJlbmNlIGxldmVsCiAgaXMgdHdvLWZvbGQgKmxvd2VyKiB0aGFuIHRoYXQgb2YgdGhlIG90aGVyIGxldmVsLAogIGEgbG9nMi1mb2xkIGNoYW5nZSBvZiAyIGluZGljYXRlcyBhIGZvdXItZm9sZCBkaWZmZXJlbmNlLAogIGEgbG9nMi1mb2xkIGNoYW5nZSBvZiAzIGluZGljYXRlcyBhbiBlaWdodC1mb2xkIGRpZmZlcmVuY2UsIGFuZCBzbyBvbi4KICAKICBTaW1pbGFybHksICpuZWdhdGl2ZSBsb2cyLWZvbGQqIHZhbHVlcyBpbmRpY2F0ZSBhIGNoYW5nZSBpbiBnZW5lIGNvdW50cyBpbiB0aGUKICBvdGhlciBkaXJlY3Rpb246IHRoZSByZWZlcmVuY2UgbGV2ZWwgaXMgKmhpZ2hlciogdGhhbiB0aGUgb3RoZXIgbGV2ZWwuCiAgCi0gVGhlIGBsZmNTRWAgY29sdW1uIGluZGljYXRlcyB0aGUgdW5jZXJ0YWludHkgaW4gdGVybXMgb2YgdGhlIHN0YW5kYXJkIGVycm9yCiAgKFNFKSBvZiB0aGUgbG9nMi1mb2xkIGNoYW5nZSBlc3RpbWF0ZS4KICAKLSBUaGUgKipgc3RhdGAqKiBjb2x1bW4gaW5kaWNhdGVzIHRoZSB2YWx1ZSBmb3IgdGhlIFdhbGQgdGVzdCdzIHRlc3Qgc3RhdGlzdGljLgoKLSBUaGUgKipgcHZhbHVlYCoqIGNvbHVtbiByZXBvcnRlZCB0aGUgKnVuY29ycmVjdGVkKiBwLXZhbHVlIGZyb20gdGhlIFdhbGQgdGVzdC4KCi0gQmVjYXVzZSB3ZSBhcmUgdGVzdGluZyBzaWduaWZpY2FuY2UgZm9yICptYW55KiBnZW5lcywKICB3ZSBuZWVkIHRvIGNvcnJlY3QgZm9yIG11bHRpcGxlIHRlc3RpbmcuCiAgREVTZXEyIHVzZXMgdGhlIEJlbmphbWluaS1Ib2NoYmVyZyBGYWxzZSBEaXNjb3ZlcnkgUmF0ZSAoRkRSKSBjb3JyZWN0aW9uLAogIGFuZCB0aGVzZSB2YWx1ZXMgYXJlIHJlcG9ydGVkIGluIHRoZSBjb2x1bW4gKipgcGFkamAqKiAoaS5lLiwgYWRqdXN0ZWQgcC12YWx1ZSkuCgpBIHN1bW1hcnkgb2YgdGhpcyBpbmZvcm1hdGlvbiBhYm91dCBlYWNoIGNvbHVtbiBjYW4gYmUgc2VlbiBieSBydW5uaW5nIHRoZQpgbWNvbHMoKWAgZnVuY3Rpb246CgpgYGB7cn0KbWNvbHMocmVzKQpgYGAKCiMjIyBgTkFgIHZhbHVlcyBpbiB0aGUgcmVzdWx0cyB0YWJsZQoKU29tZSB2YWx1ZXMgaW4gdGhlIHJlc3VsdHMgdGFibGUgY2FuIGJlIHNldCB0byBgTkFgIGZvciBvbmUgb2YgdGhlIGZvbGxvd2luZyByZWFzb25zOgoKLSBJZiBhIGdlbmUgY29udGFpbnMgYSBzYW1wbGUgd2l0aCBhIGNvdW50ICoqb3V0bGllcioqLAogIGJvdGggdGhlIHAtdmFsdWUgYW5kIGFkanVzdGVkIHAtdmFsdWUgd2lsbCBiZSBzZXQgdG8gYE5BYC4KICAoREVTZXEyIHBlcmZvcm1zIG91dGxpZXIgZGV0ZWN0aW9uIHVzaW5nIENvb2sncyBkaXN0YW5jZS4pCiAgCi0gSWYgYWxsIHNhbXBsZXMgaGF2ZSAqKnplcm8gY291bnRzKiogZm9yIGEgZ2l2ZW4gZ2VuZSwKICB0aGUgYGJhc2VNZWFuYCBjb2x1bW4gd2lsbCBiZSB6ZXJvLAogIGFuZCB0aGUgbG9nMi1mb2xkIGNoYW5nZSBlc3RpbWF0ZXMsCiAgcC12YWx1ZSBhbmQgYWRqdXN0ZWQgcC12YWx1ZSB3aWxsIGFsbCBiZSBzZXQgdG8gYE5BYC4KCi0gREVTZXEyIGFsc28gYXV0b21hdGljYWxseSBmaWx0ZXJzIGdlbmVzIHdpdGggYSAqKmxvdyBtZWFuIGNvdW50KioKICBpbiB0aGUgc2Vuc2UgdGhhdCBpdCBkb2VzIG5vdCBpbmNsdWRlIHRoZW0gaW4gdGhlIG11bHRpcGxlIHRlc3RpbmcgY29ycmVjdGlvbi4KICBUaGVyZWZvcmUsIGluIHN1Y2ggY2FzZXMsIHRoZSBwLXZhbHVlIHdpbGwgbm90IGJlIGBOQWAsCiAgYnV0IHRoZSAqYWRqdXN0ZWQqIHAtdmFsdWUgd2lsbCBiZS4KICAKICBCZWNhdXNlIHdlIGhhdmUgdmVyeSBsb3cgcG93ZXIgdG8gZGV0ZWN0IGRpZmZlcmVudGlhbCBleHByZXNzaW9uIGZvciBzdWNoCiAgbG93LWNvdW50IGdlbmVzLCBpdCBpcyBiZW5lZmljaWFsIHRvIHJlbW92ZSB0aGVtIHByaW9yIHRvIHRoZSBtdWx0aXBsZSB0ZXN0aW5nCiAgY29ycmVjdGlvbjogdGhhdCB3YXksIHRoZSBjb3JyZWN0aW9uIGJlY29tZXMgbGVzcyBzZXZlcmUgZm9yIHRoZSByZW1haW5pbmcgZ2VuZXMuCgpMZXQncyBzZWUgaG93IG1hbnkgZ2VuZXMgaGF2ZSBgTkFgIHAtdmFsdWVzOgoKYGBge3J9CiMgTnVtYmVyIG9mIGdlbmVzIHdpdGggTkEgcC12YWx1ZToKc3VtKGlzLm5hKHJlcyRwdmFsdWUpKQoKIyBBcyBhIHByb3BvcnRpb24gb2YgdGhlIHRvdGFsIG51bWJlciBvZiBnZW5lcyBpbiB0aGUgdGVzdDoKc3VtKGlzLm5hKHJlcyRwdmFsdWUpKSAvIG5yb3cocmVzKQpgYGAKCkFuZCBgTkFgIGFkanVzdGVkIHAtdmFsdWVzOgoKYGBge3J9CiMgTnVtYmVyIG9mIGdlbmVzIHdpdGggTkEgcC12YWx1ZToKc3VtKGlzLm5hKHJlcyRwYWRqKSkKCiMgQXMgYSBwcm9wb3J0aW9uIG9mIHRoZSB0b3RhbCBudW1iZXIgb2YgZ2VuZXMgaW4gdGhlIHRlc3Q6CnN1bShpcy5uYShyZXMkcGFkaikpIC8gbnJvdyhyZXMpCmBgYAoKPGJyPgoKLS0tLQoKIyMgREUgYW5hbHlzaXMgLS0gY29udHJhc3QgdHdvIGN1c3RvbSBncm91cHMKClVzaW5nIHRoZSBgcmVzdWx0c05hbWVzYCBmdW5jdGlvbiwKd2UgY2FuIHNlZSB3aGljaCBwYWlyd2lzZSBjb250cmFzdHMgYmV0d2VlbiBkaWZmZXJlbnQgbGV2ZWxzIG9mIHRoZSBmYWN0b3IgYXJlCmF2YWlsYWJsZSAodGhvdWdoIGl0IGlzIG5vdCBkaXNwbGF5ZWQgaW4gYSBwYXJ0aWN1bGFybHkgcmVhZGFibGUgZmFzaGlvbik6CgpgYGB7cn0KcmVzdWx0c05hbWVzKGRkcykKYGBgCgpOb3QgYWxsIHBhaXJ3aXNlIGNvbnRyYXN0cyBiZXR3ZWVuIHRoZSA1IGxldmVscyBpbiBvdXIgYGdyb3VwYCBmYWN0b3IgYXJlIGF2YWlsYWJsZQpoZXJlOiBpbnN0ZWFkLCBgY29udHJvbF9tb2NrYCwgd2hpY2ggd2Ugc2V0IGFzIHRoZSByZWZlcmVuY2UgbGV2ZWwsIGlzIGJlaW5nIGNvbXBhcmVkCndpdGggdGhlIG90aGVyIDMgbGV2ZWxzLgooSG93ZXZlciwgd2UgY2FuIG1ha2Ugb3RoZXIgcGFpcndpc2UgY29tcGFyaXNvbnMsIHRvby4pCgpBYm92ZSwgd2UgbG9va2VkIGF0IHRoZSByZXN1bHRzIGZvciB0aGUgKmxhc3QqIG9mIHRoZXNlIGNvbXBhcmlzb25zCihgZ3JvdXBfUmlfbXliX3ZzX2NvbnRyb2xfbW9ja2AsIGkuZS4gIlJpX215YiIgdnMuICJjb250cm9sX21vY2siKSwKc2ltcGx5IGJlY2F1c2UgREVTZXEyIHdpbGwgc2hvdyB0aGUgbGFzdCBjb21wYXJpc29uIGJ5IGRlZmF1bHQgd2hlbiBjYWxsaW5nCnRoZSBgcmVzdWx0cygpYCBmdW5jdGlvbi4KClRvIHNlZSB0aGUgcmVzdWx0cyB0YWJsZSBmb3Igb25lIG9mIHRoZSBvdGhlciAzIGNvbXBhcmlzb25zLAp3ZSBwYXNzIGEgdmVjdG9yIHRvIHRoZSBgY29udHJhc3RgIGFyZ3VtZW50IG9mIHRoZSBgcmVzdWx0cygpYCBmdW5jdGlvbiB3aXRoCnRoZSBmYWN0b3IgKGBncm91cGApIGFuZCB0aGUgdHdvIGxldmVscyB0byBiZSBjb250cmFzdGVkLgpGb3IgZXhhbXBsZSwgdG8gc2VlIHRoZSByZXN1bHRzIGZvciAiUmlfbW9jayIgdnMuICJjb250cm9sX21vY2siOgoKYGBge3J9CiMgSGVyZSwgd2UgY291bGQgc3BlY2lmeSAqYW55KiBwYWlyd2lzZSBjb250cmFzdCwKIyBub3QganVzdCB0aGUgb25lcyB3aXRoICJjb250cm9sX21vY2siIHRoYXQgcmVzdWx0c05hbWVzKCkgcHJpbnRzIGFzIHNlZW4gYWJvdmUuCm15X2NvbnRyYXN0IDwtIGMoIlJpX21vY2siLCAiY29udHJvbF9tb2NrIikKCnJlcyA8LSByZXN1bHRzKGRkcywKICAgICAgICAgICAgICAgY29udHJhc3QgPSBjKCJncm91cCIsIG15X2NvbnRyYXN0KSkKYGBgCgpIb3cgbWFueSBhZGp1c3RlZCBwLXZhbHVlcyB3ZXJlIGxlc3MgdGhhbiAwLjE/CgpgYGB7cn0Kc3VtKHJlcyRwYWRqIDwgMC4xLCBuYS5ybSA9IFRSVUUpCmBgYAoKV2UnbGwgYWxzbyBjcmVhdGUgYW4gb2JqZWN0IHdpdGggKmFkanVzdGVkIChzaHJ1bmtlbikgTEZDIGVzdGltYXRlcyosCndoaWNoIHdpbGwgYmUgdXNlZnVsIGZvciB2aXN1YWxpemF0aW9uIGFuZCByYW5raW5nIG9mIGdlbmVzIGJ5IExGQzoKCmBgYHtyfQpteV9jb2VmIDwtIHBhc3RlMCgiZ3JvdXBfIiwgcGFzdGUwKG15X2NvbnRyYXN0LCBjb2xsYXBzZSA9ICJfdnNfIikpCm15X2NvZWYKCnJlc19MRkMgPC0gbGZjU2hyaW5rKGRkcywKICAgICAgICAgICAgICAgICAgICAgY29lZiA9IG15X2NvZWYsCiAgICAgICAgICAgICAgICAgICAgIHR5cGUgPSAiYXBlZ2xtIiwKICAgICAgICAgICAgICAgICAgICAgbGZjVGhyZXNob2xkID0gMSkKYGBgCgpIZXJlLCB3ZSBoYWQgdG8gcHJvdmlkZSB0aGUgY29udHJhc3QgKCJjb2VmZmljaWVudCIpIGluIHRoZSBmb3JtYXQKZ2l2ZW4gYnkgYHJlc3VsdHNOYW1lcyhkZHMpYC4KKEFuZCB0aGF0IGxvb2tzIGEgYml0IGNvbmZ1c2luZyBiZWNhdXNlIHRoaXMgZm9ybWF0IHVzZXMgdW5kZXJzY29yZXMgdG8gc2VwYXJhdGUgbGV2ZWxzLAp3aGlsZSBvdXIgZmFjdG9yIGxldmVscyB0aGVtc2VsdmVzIGFsc28gY29udGFpbiB1bmRlcnNjb3Jlcy4pCgpXZSBhbHNvIHNwZWNpZmllZCBhIHRocmVzaG9sZCBvZiAxIGZvciB0aGUgTEZDIHZhbHVlIChgbGZjVGhyZXNob2xkID0gMWApLApzbyB3ZSBnZXQgcy12YWx1ZXMgKGFuYWxvZ291cyB0byBwLXZhbHVlcykgdGhhdCB0ZXN0IG5vdCBmb3IgZGlmZmVyZW50aWFsCmV4cHJlc3Npb24gb2YgYW55IG1hZ25pdHVkZSAoYXMgaW4gdGhlIHRlc3RzIGFib3ZlKSwKYnV0IHdoZXRoZXIgdGhlIExGQyBpcyBncmVhdGVyIHRoYW4gb3VyIHNwZWNpZmllZCB0aHJlc2hvbGQ6IAoKYGBge3J9CmhlYWQocmVzX0xGQykKCnN1bShyZXNfTEZDJHN2YWx1ZSA8IDAuMSwgbmEucm0gPSBUUlVFKQpgYGAKClRoaXMgY2FuIGJlIGEgdXNlZnVsIHdheSB0byB0cnkgYW5kIHRlYXNlIG91dCBzdGF0aXN0aWNhbCBmcm9tIGJpb2xvZ2ljYWwgc2lnbmlmaWNhbmNlLgoKPGJyPgoKLS0tLQoKIyMgVmlzdWFsbHkgZXhwbG9yaW5nIHRoZSByZXN1bHRzCgpXZSB3aWxsIGNyZWF0ZSBhIGZldyBwbG90cywgYnkgd2F5IG9mIGV4YW1wbGUsCm9mIHRoZSByZXN1bHRzIGZvciB0aGUgIlJpX21vY2siIHZlcnN1cyAiY29udHJvbF9tb2NrIiBjb21wYXJpc29uLAp3aGljaCB3ZSBleHRyYWN0ZWQgYWJvdmUuCgojIyMgTUEtcGxvdDoKCkZvciBhIG5pY2Ugb3ZlcnZpZXcgb2YgdGhlIHJlc3VsdHMsIHdlIGNhbiBwbG90IGEgc28tY2FsbGVkICJNQSBwbG90Ii4KQW4gTUEgcGxvdCBzaG93cywgZm9yIGVhY2ggZ2VuZToKCi0gQ291bnQgZGlmZmVyZW5jZXMgaW4gdGVybXMgb2YgTEZDIGJldHdlZW4gdHdvIGdyb3Vwcywgb24gdGhlIHktYXhpcy4KCi0gTWVhbiBjb3VudHMgYWNyb3NzIGJvdGggZ3JvdXBzLCBvbiB0aGUgeC1heGlzLgoKV2UgY2FuIGNyZWF0ZSBhbiBNQSBwbG90IHVzaW5nIERFU2VxMidzIGBwbG90TUFgIGZ1bmN0aW9uLAp3aXRoIHNpZ25pZmljYW50bHkgZGlmZmVyZW50aWFsbHkgZXhwcmVzc2VkIGdlbmVzIGRpc3BsYXllZCBpbiBibHVlOgoKYGBge3J9CnBsb3RNQShyZXMsIHlsaW0gPSBjKC01LCA1KSkKYGBgCgpUbyBiZSBhYmxlIHRvIGN1c3RvbWl6ZSB0aGUgcGxvdCwKd2UnbGwgdXNlIGByZXR1cm5EYXRhID0gVFJVRWAgbGlrZSB3ZSBoYXZlIGRvbmUgd2l0aCBwcmV2aW91cyBwbG90cywKYW5kIHRoZW4gcGxvdCB0aGUgcmVzdWx0aW5nIGRhdGFmcmFtZSB3aXRoICpnZ3Bsb3QyKjoKCmBgYHtyfQpkIDwtIHBsb3RNQShyZXMsIHJldHVybkRhdGEgPSBUUlVFKQoKZ2dwbG90KGQsIGFlcyh4ID0gbWVhbiwgeSA9IGxmYywgY29sb3IgPSBpc0RFKSkgKwogIGdlb21fcG9pbnQoc2l6ZSA9IDAuNSkgKwogIHNjYWxlX3hfbG9nMTAoKSArCiAgc2NhbGVfeV9jb250aW51b3VzKGxpbWl0cyA9IGMoLTEwLCAxMCkpICsKICBzY2FsZV9jb2xvcl9tYW51YWwodmFsdWVzID0gYygnZ3JleTUwJywgJ2JsdWUnKSkgKwogIGd1aWRlcyhjb2xvciA9IEZBTFNFKSArCiAgbGFicyh4ID0gIk1lYW4gb2Ygbm9ybWFsaXplZCBjb3VudHMiLAogICAgICAgeSA9ICJMRkMiKQpgYGAKCldlIGNhbiBzZWUgdGhhdCBsb3dseS1leHByZXNzZWQgZ2VuZXMgdGVuZCB0byBkZXZpYXRlIGZyb20gYW4gTEZDIG9mIDAKKHNhbWUgbWVhbiBleHByZXNzaW9uIGxldmVscyBpbiB0aGUgdHdvIGdyb3VwcykgbXVjaCBtb3JlIHRoYW4KaGlnaGx5LWV4cHJlc3NlZCBnZW5lcyBkby4KSG93ZXZlciwgdGhpcyBpcyBhbiBhcnRpZmFjdCBvZiBub2lzZSBvdmVycG93ZXJpbmcgdGhlIHNpZ25hbCB3aGVuCmV4cHJlc3Npb24gdmFsdWVzIGFyZSBsb3cuCldlIGNhbiBhbHNvIHNlZSB0aGF0IG5vIGdlbmVzIGluIHRoZSBmYXIgbGVmdCBwYXJ0IG9mIHRoZSBwbG90IGFyZQpkaWZmZXJlbnRpYWxseSBleHByZXNzZWQ6IHRoaXMgaXMgZHVlIHRvIHRoaXMgc2FtZSBsYWNrIG9mIHBvd2VyLgoKREVTZXEyIHByb3ZpZGVzIHNldmVyYWwgbWV0aG9kcyB0byBhZGp1c3QgTEZDIGVzdGltYXRlcyBmb3IgdGhpcyBsb3ctZXhwcmVzc2lvbgpiaWFzLgpXZSB1c2VkIG9uZSBvZiB0aG9zZSAoYGxmY1NocmluaygpYCkgYWJvdmUgdG8gcHJvZHVjZSB0aGUgYHJlc19MRkNgIG9iamVjdC4KTGV0J3MgY3JlYXRlIGFub3RoZXIgTUEgcGxvdCB3aXRoIHRoZXNlIGFkanVzdGVkIExGQyBlc3RpbWF0ZXM6CgpgYGB7cn0KZCA8LSBwbG90TUEocmVzX0xGQywgeWxpbSA9IGMoLTUsIDUpLCByZXR1cm5EYXRhID0gVFJVRSkKCmdncGxvdChkLCBhZXMoeCA9IG1lYW4sIHkgPSBsZmMsIGNvbG9yID0gaXNERSkpICsKICBnZW9tX3BvaW50KHNpemUgPSAwLjUpICsKICBzY2FsZV94X2xvZzEwKCkgKwogIHNjYWxlX2NvbG9yX21hbnVhbCh2YWx1ZXMgPSBjKCdncmV5NTAnLCAnYmx1ZScpKSArCiAgZ3VpZGVzKGNvbG9yID0gRkFMU0UpICsKICBsYWJzKHggPSAiTWVhbiBvZiBub3JtYWxpemVkIGNvdW50cyIsCiAgICAgICB5ID0gIlNocnVua2VuIExGQyIpCmBgYAoKTm90ZSB0aGF0IHNpZ25pZmljYW5jZSBpcyBub3cgc3BlY2lmaWVkIGZvciBMRkMgPiAxIC0tIEZJTklTSAoKRmluYWxseSwgZm9yIGEgcGxvdCBsaWtlIHRoaXMsCml0IGNvdWxkIGJlIHVzZWZ1bCB0byBiZSBhYmxlIHRvIGlkZW50aWZ5IGluZGl2aWR1YWwgZ2VuZXMuClRoZXJlIGFyZSB3YXkgdG9vIG1hbnkgdG8gcHJpbnQgdGhlIGdlbmUgbmFtZXMgb24gdGhlIHBsb3QsIHRob3VnaC4KSW5zdGVhZCwgd2UgY2FuIGFsc28gbWFrZSB0aGUgcGxvdCAqKmludGVyYWN0aXZlIHdpdGggUGxvdGx5KiosCnNvIHdlIGNhbiBzZWUgdGhlIGlkZW50aXR5IG9mIGVhY2ggZ2VuZSAqd2hlbiB3ZSBob3ZlciBvdmVyIHRoZSBwb2ludCo6CgpgYGB7cn0KbGlicmFyeShwbG90bHkpCgojIFRvIHNob3cgdGhlIGdlbmUgbmFtZSwgd2UgbmVlZCB0byBoYXZlIGEgY29sdW1uIHdpdGggZ2VuZSBuYW1lcy4KIyBDdXJyZW50bHksIHRoZSBnZW5lIG5hbWVzIGFyZSByb3cgbmFtZXMgYnV0IGdncGxvdDIgKGFuZCBvdGhlciB0aWR5dmVyc2UKIyBhcHBsaWNhdGlvbnMpIGRvbid0IGxpa2UgdGhhdCwgc28gd2UgY3JlYXRlIGEgY29sdW1uIHdpdGggZ2VuZSBuYW1lczoKZCRnZW5lIDwtIHJvd25hbWVzKGQpCgojIEZpcnN0IHdlIGNyZWF0ZSBhIHZlcnkgc2ltaWxhciBnZ3Bsb3QgdG8gd2hhdCB3ZSBkaWQgYWJvdmUsCiMgYnV0IHdlIGFzc2lnbiBnZW5lIG5hbWVzIHRvICJ0ZXh0IjoKcF9tYSA8LSBnZ3Bsb3QoZCwKICAgICAgICAgICAgICAgYWVzKHggPSBtZWFuLCB5ID0gbGZjLCBjb2xvciA9IGlzREUsIHRleHQgPSBnZW5lKSkgKwogIGdlb21fcG9pbnQoc2l6ZSA9IDAuNSkgKwogIHNjYWxlX3hfbG9nMTAoKSArCiAgc2NhbGVfY29sb3JfbWFudWFsKHZhbHVlcyA9IGMoJ2dyZXk1MCcsICdibHVlJykpICsKICBndWlkZXMoY29sb3IgPSBGQUxTRSkgKwogIGxhYnMoeCA9ICJNZWFuIG9mIG5vcm1hbGl6ZWQgY291bnRzIiwKICAgICAgIHkgPSAiU2hydW5rZW4gTEZDIikKCiMgRmluYWxseSwgd2UgbWFrZSB0aGUgcGxvdCBpbnRlcmFjdGl2ZSBhbmQgdGVsbCBQbG90bHkgdGhhdCBpdCBzaG91bGQgc2hvdwojIHRoZSAidGV4dCIgKGkuZS4gZ2VuZSBuYW1lcykgYXMgdGhlICJ0b29sdGlwIiwgbWVhbmluZyB1cG9uIGhvdmVyaW5nOgpnZ3Bsb3RseShwX21hLCB0b29sdGlwID0gInRleHQiKQpgYGAKCiMjIyBQbG90IHNwZWNpZmljIGdlbmVzCgpXZSBjYW4gYWxzbyBjcmVhdGUgcGxvdCBvZiBleHByZXNzaW9uIGxldmVscyBmb3IgaW5kaXZpZHVhbCBnZW5lcy4KVGhhdCBpcyBlc3BlY2lhbGx5IGludGVyZXN0aW5nIGZvciBnZW5lcyB3aXRoIGhpZ2hseSBzaWduaWZpY2FudCBkaWZmZXJlbnRpYWwKZXhwcmVzc2lvbi4KCkxldCdzIHBsb3QgdGhlIHRvcC01IG1vc3Qgc2lnbmlmaWNhbnRseSBkaWZmZXJlbnRpYWxseSBleHByZXNzZWQgZ2VuZXM6CgpgYGB7ciwgZXZhbD1UUlVFfQojIEZpcnN0LCB3ZSBzZWxlY3QgdGhlIDUgZ2VuZXMgd2l0aCB0aGUgbG93ZXN0IGFkanVzdGVkIHAtdmFsdWU6CnRvcDUgPC0gcm93Lm5hbWVzKHJlc1tvcmRlcihyZXMkcGFkailbMTo1XSwgXSkKYGBgCgpgYGB7ciwgZXZhbD1UUlVFfQojIFRoZW4gd2UgY3JlYXRlIGEgZnVuY3Rpb24gdG8gbWFrZSBhIHBsb3QgZm9yIGEgc2luZ2xlIGdlbmU6CnBsb3RnZW5lIDwtIGZ1bmN0aW9uKGdlbmVJRCwgZGRzKSB7CiAgCiAgZCA8LSBwbG90Q291bnRzKGRkcywKICAgICAgICAgICAgICAgICAgZ2VuZSA9IGdlbmVJRCwKICAgICAgICAgICAgICAgICAgaW50Z3JvdXAgPSAiZ3JvdXAiLAogICAgICAgICAgICAgICAgICByZXR1cm5EYXRhID0gVFJVRSkKCiAgcCA8LSBnZ3Bsb3QoZCwgYWVzKHggPSBncm91cCwgeSA9IGNvdW50KSkgKwogICAgICAgICAgZ2VvbV9wb2ludChwb3NpdGlvbiA9IHBvc2l0aW9uX2ppdHRlcih3ID0gMC4xLCBoID0gMCkpICsKICAgICAgICAgIGxhYnModGl0bGUgPSBnZW5lSUQpICsKICAgICAgICAgIHRoZW1lX2J3KCkKICAKICBwcmludChwKQp9CmBgYAoKRmluYWxseSwgd2UgdXNlIGBzYXBwbHkoKWAgdG8gYXBwbHkgdGhpcyBmdW5jdGlvbiB0byBlYWNoIG9mIG91ciBnZW5lcyBpbiB0aGUKYHRvcDVgIHZlY3Rvci4KCmBgYHtyfQpub25lIDwtIHNhcHBseSh0b3A1LCBwbG90Z2VuZSwgZGRzKQpgYGAKCklmIHdlIHdhbnRlZCB0bywgd2UgY291bGQgZWFzaWx5IGNyZWF0ZSBwbG90cyBmb3IgMTAwcyBvZiBnZW5lcywgdGhpcyB3YXkuCgojIyMgSGVhdG1hcAoKV2UgY2FuIGNyZWF0ZSBoZWF0bWFwcyB3aXRoIHRoZSBgcGhlYXRtYXBgIGZ1bmN0aW9uLgpMZXQncyBzdGFydCBieSBjcmVhdGluZyBhIGZ1bmN0aW9uIHRoYXQgd2lsbCBwbG90IGEgaGVhdG1hcCBnaXZlbiBhIHZlY3RvcgpvZiBnZW5lIElEcyBhbmQgYSBERVNlcTIgb2JqZWN0IGBkZHNgOgoKYGBge3IsIGV2YWw9VFJVRX0KcGxvdF9oZWF0bWFwIDwtIGZ1bmN0aW9uKGdlbmVJRHMsIGRkcykgewogIAogIG50ZCA8LSBhc3NheShub3JtVHJhbnNmb3JtKGRkcykpCiAgCiAgbnRkX3NlbCA8LSBudGRbbWF0Y2goZ2VuZUlEcywgcm93bmFtZXMobnRkKSksIF0KICBkZl9tZXRhIDwtIGFzLmRhdGEuZnJhbWUoY29sRGF0YShkZHMpWywgYygiQU1GIiwgIlRyZWF0bWVudCIpXSkKICAKICBwaGVhdG1hcChudGRfc2VsLAogICAgICAgICBjbHVzdGVyX3Jvd3MgPSBGQUxTRSwKICAgICAgICAgY2x1c3Rlcl9jb2xzID0gRkFMU0UsCiAgICAgICAgIHNob3dfcm93bmFtZXMgPSBGQUxTRSwKICAgICAgICAgYW5ub3RhdGlvbl9jb2wgPSBkZl9tZXRhKQp9CmBgYAoKTm93LCB3ZSBjYW4gZWFzaWx5IGNyZWF0ZSBhIGhlYXRtYXAgZm9yIHRoZSB0b3AtMjAgbW9zdCBoaWdobHkgZGlmZmVyZW50aWFsbHkKZXhwcmVzc2VkIGdlbmVzOgoKYGBge3IsIGV2YWw9VFJVRX0KdG9wMjBfREUgPC0gcm93Lm5hbWVzKHJlc1tvcmRlcihyZXMkcGFkailbMToyMF0sIF0pCnBsb3RfaGVhdG1hcCh0b3AyMF9ERSwgZGRzKQpgYGAKCk9yIGZvciB0aGUgMjAgbW9zdCBoaWdobHkgZXhwcmVzc2VkIGdlbmVzOgoKYGBge3IsIGV2YWw9VFJVRX0KdG9wMjBfaGlfaWR4IDwtIG9yZGVyKHJvd01lYW5zKGNvdW50cyhkZHMsIG5vcm1hbGl6ZWQgPSBUUlVFKSksCiAgICAgICAgICAgICAgICAgIGRlY3JlYXNpbmcgPSBUUlVFKVsxOjIwXQp0b3AyMF9oaSA8LSByb3cubmFtZXMoZGRzKVt0b3AyMF9oaV9pZHhdCnBsb3RfaGVhdG1hcCh0b3AyMF9oaSwgZGRzKQpgYGAKCiMjIyBFeHBvcnQgdGhlIHJlc3VsdHMKCkxldCdzIHNhdmUgdGhlIHJlc3VsdHMgZGF0YWZyYW1lIHRvIGZpbGUuCgpOb3RlIHRoYXQgaXQgd2lsbCBvbmx5IGNvbnRhaW4gdGhlIHJlc3VsdHMgZm9yIG9uZSBjb21wYXJpc29uLgpBbHNvLCBpZiB3ZSB3cml0ZSB0aGUgcmVzdWx0cyBkYXRhZnJhbWUgdG8gZmlsZSwKd2Ugd29uJ3QgYmUgYWJsZSB0byB0ZWxsIGZyb20gdGhlIGZpbGUgd2hhdCB0aGUgY29tcGFyaXNvbiBpcywgc28gbGV0J3Mgc3RvcmUKdGhhdCBpbiB0d28gY29sdW1uczoKCmBgYHtyfQpteV9jb250cmFzdAoKbXlfY29udHJhc3RfcGFzdGVkIDwtIHBhc3RlMChteV9jb250cmFzdCwgY29sbGFwc2UgPSAiX3ZzXyIpCm15X2NvbnRyYXN0X3Bhc3RlZApgYGAKCmBgYHtyfQpyZXMkbGV2ZWwxIDwtIG15X2NvbnRyYXN0WzFdCnJlcyRsZXZlbDIgPC0gbXlfY29udHJhc3RbMl0Ka2FibGUoaGVhZChyZXMpKQpgYGAKCk5vdyB3ZSBjYW4gd3JpdGUgYHJlc2AgdG8gZmlsZToKCmBgYHtyLCBldmFsPUZBTFNFfQpyZXNfZmlsZSA8LSBmaWxlLnBhdGgob3V0ZGlyLCBwYXN0ZTAobXlfY29udHJhc3RfcGFzdGVkLCAnX2FsbC1yZXMudHh0JykpCgp3cml0ZS50YWJsZShyZXMsIHJlc19maWxlLAogICAgICAgICAgICBzZXAgPSAnXHQnLCByb3cubmFtZXMgPSBUUlVFLCBxdW90ZSA9IEZBTFNFKQpgYGAKCldlIG1heSBhbHNvIHdhbnQgdG8gc2F2ZSBhIHNlcGFyYXRlIHRhYmxlIHdpdGggb25seSBzaWduaWZpY2FudCByZXN1bHRzOgoKYGBge3IsIGV2YWw9RkFMU0V9CnJlc19zaWdfZmlsZSA8LSBmaWxlLnBhdGgob3V0ZGlyLCBwYXN0ZTAobXlfY29udHJhc3RfcGFzdGVkLCAnX3NpZy1yZXMudHh0JykpCgpyZXNfc2lnIDwtIHJlcyAlPiUKICBhcy5kYXRhLmZyYW1lKCkgJT4lIAogIGRwbHlyOjpmaWx0ZXIocGFkaiA8IDAuMSkKCndyaXRlLnRhYmxlKHJlc19zaWcsIHJlc19zaWdfZmlsZSwKICAgICAgICAgICAgc2VwID0gJ1x0Jywgcm93Lm5hbWVzID0gVFJVRSwgcXVvdGUgPSBGQUxTRSkKYGBgCgo8YnI+CgotLS0tCgojIyBERSBhbmFseXNpcyAtLSBhbGwgcGFpcndpc2UgY29tcGFyaXNvbnMKClRvIHJ1biB0aGUgREUgYW5hbHlzaXMgZm9yIGFsbCBwYWlyd2lzZSBjb21wYXJpc29ucywKd2Ugd2lsbCBzdGFydCBieSB3cml0aW5nIGEgZnVuY3Rpb24gdGhhdCB0YWtlcyBhIGNvbXBhcmlzb24gKGNvbnRyYXN0KSBpbiB0aGUKZm9ybSBvZiBgYyhsZXZlbDEsIGxldmVsMilgIGFsb25nIHdpdGggYSBERVNlcTIgb2JqZWN0IChgZGRzYCksCmFuZCBvdXRwdXQgYSBkYXRhIGZyYW1lIHdpdGggc2lnbmlmaWNhbnRseSBkaWZmZXJlbnRpYWxseSBleHByZXNzZWQgZ2VuZXM6CgpgYGB7cn0Kc2lnX2NvbnRyYXN0IDwtIGZ1bmN0aW9uKG15X2NvbnRyYXN0LCBkZHMpIHsKICAKICByZXNfc2lnIDwtIHJlc3VsdHMoZGRzLAogICAgICAgICAgICAgICAgICAgICBjb250cmFzdCA9IGMoImdyb3VwIiwgbXlfY29udHJhc3QpKSAlPiUKICAgIGFzLmRhdGEuZnJhbWUoKSAlPiUgCiAgICBkcGx5cjo6ZmlsdGVyKHBhZGogPCAwLjEpICU+JQogICAgbXV0YXRlKGxldmVsMSA9IG15X2NvbnRyYXN0WzFdLAogICAgICAgICAgIGxldmVsMiA9IG15X2NvbnRyYXN0WzJdKQogIAogIGNhdChteV9jb250cmFzdFsxXSwgInZlcnN1cyIsIG15X2NvbnRyYXN0WzJdLCAiOiIsIG5yb3cocmVzX3NpZyksICJzaWduaWZpY2FudFxuIikKICAKICByZXR1cm4ocmVzX3NpZykKfQpgYGAKClRoZW4sIHdlIHdpbGwgY3JlYXRlIGEgbGlzdCB3aXRoIGFsbCBwYWlyd2lzZSBjb21iaW5hdGlvbnMgb2Ygb3VyIGBncm91cGAgZmFjdG9yCnVzaW5nIHRoZSBgY29tYm4oKWAgZnVuY3Rpb246CgpgYGB7cn0KY29tcHMgPC0gY29tYm4obGV2ZWxzKGRkc0Bjb2xEYXRhJGdyb3VwKSwgMiwgc2ltcGxpZnkgPSBGQUxTRSkKYGBgCgpGaW5hbGx5LCB3ZSBydW4gdGhlIGZ1bmN0aW9uIGZvciBhbGwgb2Ygb3VyIHBhaXJ3aXNlIGNvbXBhcmlzb25zLAphbmQgdXNpbmcgYGRvLmNhbGwocmJpbmQsIC4uLilgLCB3ZSBjb25jYXRlbmF0ZSBhbGwgdGhlIHJlc3VsdHMgaW4gYSBzaW5nbGUKZGF0YSBmcmFtZToKCmBgYHtyfQpzaWdfYWxsX2NvbnRyYXN0cyA8LSBkby5jYWxsKHJiaW5kLCBsYXBwbHkoY29tcHMsIHNpZ19jb250cmFzdCwgZGRzKSkKCiMgVGhlIGBrYWJsZSgpYCBmdW5jdGlvbiBpcyBqdXN0IHRvIGRpc3BsYXkgdGFibGVzIG5pY2VseSBpbiB0aGlzIFIgTWFya2Rvd24KIyBkb2N1bWVudCAtLSB1c2UgYGhlYWQoc2lnX2FsbF9jb250cmFzdHMpYCBpbiB5b3VyIG93biBjb2RlLgprYWJsZShoZWFkKHNpZ19hbGxfY29udHJhc3RzKSkKYGBgCgotLS0tCgojIyBERSBhbmFseXNpcyAtLSB3aXRoIHR3byBmYWN0b3JzCgojIyMgQ29udHJvbGxpbmcgZm9yIG9uZSBmYWN0b3IKClNheSB3ZSB3YW50ZWQgdG8gYW5hbHl6ZSB0aGUgZWZmZWN0IG9mICJtb2NrIiB2ZXJzdXMgIm15YiIKKCJUcmVhdG1lbnQiIGNvbHVtbikgd2hpbGUgY29udHJvbGxpbmcgZm9yIHRoZSBlZmZlY3RzIG9mICJjb250cm9sIiB2ZXJzdXMgIlJpIgooIkFNRiIgY29sdW1uKS4KCkxldCdzIHN0YXJ0IGJ5IHR1cm5pbmcgIlRyZWF0bWVudCIgYW5kICJBTUYiIGludG8gZmFjdG9ycywKYW5kIHNhdmluZyBhIG5ldyBERVNlcTIgb2JqZWN0OgoKYGBge3J9CiMgQ29udmVydCBUcmVhdG1lbnQgYW5kIEFNRiBpbnRvIGZhY3RvcnM6IApkZHNfcmF3JFRyZWF0bWVudCA8LSByZWxldmVsKGZhY3RvcihkZHNfcmF3JFRyZWF0bWVudCksIHJlZiA9ICJtb2NrIikKZGRzX3JhdyRBTUYgPC0gcmVsZXZlbChmYWN0b3IoZGRzX3JhdyRBTUYpLCByZWYgPSAiY29udHJvbCIpCgojIFNhdmUgYSBuZXcgb2JqZWN0OgpkZHNfMmZfcmF3IDwtIGRkc19yYXcKYGBgCgpUbyBpbmNsdWRlIGJvdGggZmFjdG9ycywgd2UgdXNlIGEgYCtgIGluIHRoZSBmb3JtdWxhLgpOb3RlIHRoYXQgdGhlIG9yZGVyIG1hdHRlcnM6CnVzaW5nIGBBTUYgKyBUcmVhdG1lbnRgLCB3ZSB0ZXN0IGZvciB0aGUgZWZmZWN0IG9mICJUcmVhdG1lbnQiICh0aGUgbGFzdCBmYWN0b3IpLAp3aGlsZSBjb250cm9sbGluZyBmb3IgdGhlIGVmZmVjdCBvZiBBTUYgKHRoZSBmaXJzdCBmYWN0b3IpLgoKYGBge3J9CmRlc2lnbihkZHNfMmZfcmF3KSA8LSBmb3JtdWxhKH4gQU1GICsgVHJlYXRtZW50KQpgYGAKClJ1biBERVNlcSB3aXRoIHRoZSBuZXcgZGVzaWduOgoKYGBge3J9CmRkc18yZiA8LSBERVNlcShkZHNfMmZfcmF3KQpyZXMgPC0gcmVzdWx0cyhkZHNfMmYpCmBgYAoKSG93IG1hbnkgYWRqdXN0ZWQgcC12YWx1ZXMgd2VyZSBsZXNzIHRoYW4gMC4xPwoKYGBge3J9CnN1bShyZXMkcGFkaiA8IDAuMSwgbmEucm0gPSBUUlVFKQpgYGAKCiMjIyBXaXRoIGFuIGludGVyYWN0aW9uIHRlcm0KCldlIGNhbiBhZGQgYW4gaW50ZXJhY3Rpb24gdGVybSB1c2luZyBub3RhdGlvbiBsaWtlIGBsZXZlbDE6bGV2ZWwyYCwgd2l0aCBhIGNvbG9uLgpJbiB0aGlzIGNhc2UsIHdlIHdhbnQgdG8gaW5jbHVkZSBib3RoICJBTUYiLCAiVHJlYXRtZW50IiwgYW5kIHRoZSBpbnRlcmFjdGlvbgpiZXR3ZWVuIHRoZSB0d286CgpgYGB7cn0KIyBTYXZlIGEgbmV3IG9iamVjdDoKZGRzXzJmaV9yYXcgPC0gZGRzX3JhdwoKIyBUaGUgaW50ZXJhY3Rpb24gdGVybSBpcyBgQU1GOlRyZWF0bWVudGA6CmRlc2lnbihkZHNfMmZpX3JhdykgPC0gZm9ybXVsYSh+IEFNRiArIFRyZWF0bWVudCArIEFNRjpUcmVhdG1lbnQpCmBgYAoKUnVuIERFU2VxIHdpdGggdGhlIG5ldyBkZXNpZ246CgpgYGB7cn0KZGRzXzJmaSA8LSBERVNlcShkZHNfMmZpX3JhdykKCnJlc3VsdHNOYW1lcyhkZHNfMmZpKQpgYGAKCkZyb20gdGhlIFtERVNlcTIgdmlnbmV0dGVdKGh0dHA6Ly9iaW9jb25kdWN0b3Iub3JnL3BhY2thZ2VzL2RldmVsL2Jpb2MvdmlnbmV0dGVzL0RFU2VxMi9pbnN0L2RvYy9ERVNlcTIuaHRtbCNpbnRlcmFjdGlvbnMpOgo+VGhlIGtleSBwb2ludCB0byByZW1lbWJlciBhYm91dCBkZXNpZ25zIHdpdGggaW50ZXJhY3Rpb24gdGVybXMgaXMgdGhhdCwgdW5saWtlIGZvciBhIGRlc2lnbiBgfiBnZW5vdHlwZSArIGNvbmRpdGlvbmAsIHdoZXJlIHRoZSBjb25kaXRpb24gZWZmZWN0IHJlcHJlc2VudHMgdGhlIG92ZXJhbGwgZWZmZWN0IGNvbnRyb2xsaW5nIGZvciBkaWZmZXJlbmNlcyBkdWUgdG8gZ2Vub3R5cGUsIGJ5IGFkZGluZyBgZ2Vub3R5cGU6Y29uZGl0aW9uYCwgdGhlIG1haW4gY29uZGl0aW9uIGVmZmVjdCBvbmx5IHJlcHJlc2VudHMgdGhlIGVmZmVjdCBvZiBjb25kaXRpb24gZm9yIHRoZSByZWZlcmVuY2UgbGV2ZWwgb2YgZ2Vub3R5cGUgKEksIG9yIHdoaWNoZXZlciBsZXZlbCB3YXMgZGVmaW5lZCBieSB0aGUgdXNlciBhcyB0aGUgcmVmZXJlbmNlIGxldmVsKS4gVGhlIGludGVyYWN0aW9uIHRlcm1zIGdlbm90eXBlSUkuY29uZGl0aW9uQiBhbmQgZ2Vub3R5cGVJSUkuY29uZGl0aW9uQiBnaXZlIHRoZSBkaWZmZXJlbmNlIGJldHdlZW4gdGhlIGNvbmRpdGlvbiBlZmZlY3QgZm9yIGEgZ2l2ZW4gZ2Vub3R5cGUgYW5kIHRoZSBjb25kaXRpb24gZWZmZWN0IGZvciB0aGUgcmVmZXJlbmNlIGdlbm90eXBlLgoKTm93LCB3ZSBjYW4gY2hlY2sgYSBudW1iZXIgb2YgZGlmZmVyZW50IGVmZmVjdHM6CgojIyMjIERvZXMgdGhlIGVmZmVjdHMgb2YgVHJlYXRtZW50IGRpZmZlciBhbW9uZyBsZXZlbHMgb2YgQU1GPwoKVGhpcyBpcyBkZXNjcmliZWQgYnkgdGhlIGludGVyYWN0aW9uIHRlcm0sIHdoaWNoIGNhbWUgbGFzdCBpbiBvdXIgZm9ybXVsYS4KVGhlcmVmb3JlLCB0aGUgYHJlc3VsdHMoKWAgZnVuY3Rpb24gd2lsbCBnZXQgcmVzdWx0cyBmb3IgdGhlIGludGVyYWN0aW9uIHRlcm0gYnkgZGVmYXVsdDoKCmBgYHtyfQpyZXMgPC0gcmVzdWx0cyhkZHNfMmZpKQprYWJsZShoZWFkKHJlcykpCmBgYAoKIyMjIyBUaGUgZWZmZWN0IG9mIFRyZWF0bWVudCBpbiAiY29udHJvbCIgQU1GOiAKCmBgYHtyfQpyZXMgPC0gcmVzdWx0cyhkZHNfMmZpLAogICAgICAgICAgICAgICBjb250cmFzdCA9IGMoIlRyZWF0bWVudCIsICJteWIiLCAibW9jayIpKQprYWJsZShoZWFkKHJlcykpCmBgYAoKIyMjIyBUaGUgZWZmZWN0IG9mIFRyZWF0bWVudCBpbiAibXliIiBBTUY6IAoKYGBge3J9CnJlcyA8LSByZXN1bHRzKGRkc18yZmksCiAgICAgICAgICAgICAgIGNvbnRyYXN0ID0gbGlzdChjKCJUcmVhdG1lbnRfbXliX3ZzX21vY2siLCAiQU1GUmkuVHJlYXRtZW50bXliIikpKQprYWJsZShoZWFkKHJlcykpCmBgYAoKIyMjIyBUaGUgZWZmZWN0IG9mIEFNRiBpbiB0aGUgIm1vY2siIFRyZWF0bWVudDoKCmBgYHtyfQpyZXMgPC0gcmVzdWx0cyhkZHNfMmZpLAogICAgICAgICAgICAgICAgY29udHJhc3QgPSBjKCJBTUYiLCAiUmkiLCAiY29udHJvbCIpKQprYWJsZShoZWFkKHJlcykpCmBgYAoKIyMjIyBUaGUgZWZmZWN0IG9mIEFNRiBpbiAibXliIiBUcmVhdG1lbnQ6CgpgYGB7cn0KcmVzIDwtIHJlc3VsdHMoZGRzXzJmaSwKICAgICAgICAgICAgICAgY29udHJhc3QgPSBsaXN0KGMoIkFNRl9SaV92c19jb250cm9sIiwgIkFNRlJpLlRyZWF0bWVudG15YiIpKSkKa2FibGUoaGVhZChyZXMpKQpgYGAK